Method and system for uniformly scheduling heterogeneous storage resources across cloud platforms

Abstract

The invention provides a cross-cloud platform heterogeneous storage resource unified scheduling method and system, and relates to the technical field of distributed storage, the method comprises the following steps: firstly obtaining heterogeneous storage resource information and virtual machine storage demand information of a plurality of cloud platforms, the resource information comprising a tagged data set; performing digital processing on the tagged data set, generating a feature vector set and constructing a resource topological graph; a resource scheduling algorithm is adopted to carry out collaborative analysis on the feature vector set, the topological graph and the storage demand, target heterogeneous resources are calculated, and a cross-cloud storage configuration scheme is generated; performing virtualization processing on the target resource based on the scheme to form a virtual storage pool; and finally, uniformly managing and distributing the resources in the pool according to needs through a load-aware scheduling method, thereby realizing uniform scheduling of heterogeneous storage resources across cloud platforms. According to the method, the scheduling efficiency and the resource utilization rate of heterogeneous storage resources in a cross-cloud environment are improved.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular to a unified scheduling method and system for heterogeneous storage resources across cloud platforms. Background Technology

[0002] As enterprises accelerate their digital transformation, multi-cloud architecture is gradually becoming the mainstream IT deployment model. Against this backdrop, in the face of the need to manage heterogeneous storage resources in cross-cloud environments, enterprises need to uniformly schedule different types of storage resources distributed across multiple cloud platforms to meet the dynamic needs of virtual machines for storage resources. This scenario not only requires the system to be able to perceive the status and characteristics of storage resources on each cloud platform in real time, but also to comprehensively consider multi-dimensional constraints such as performance matching, geographical distribution, and network latency, so as to achieve intelligent allocation and efficient utilization of storage resources.

[0003] Currently, a cross-cloud storage scheduling scheme based on resource tag matching is being used to address this challenge. This scheme adds standardized descriptive tags to storage resources on different cloud platforms and matches and filters them based on the storage requirement tags proposed by virtual machines. Specifically, the scheme first collects storage resource tag information from various cloud platforms, which covers key features such as storage type, performance level, and geographical location. After obtaining these features, the initial screening of resources can be completed based on the tag conditions specified in the virtual machine's requirements. Subsequently, a weighted scoring algorithm is used to calculate and compare the candidate resources, thereby finally selecting the storage resource with the highest comprehensive score for allocation.

[0004] However, this scheme mainly relies on a static label matching mechanism during resource scheduling, lacking the ability to effectively perceive the dynamic relationships between resources, which may lead to network transmission bottlenecks during resource selection. At the same time, the static weight allocation mechanism adopted by the scheme is also difficult to flexibly adapt to changing application load scenarios, resulting in insufficient performance in resource utilization optimization. In addition, due to the relatively limited support for performance isolation and quality of service assurance mechanisms between storage resources, this scheme may further affect the performance stability of critical business operations. Summary of the Invention

[0005] This application provides a unified scheduling method and system for heterogeneous storage resources across cloud platforms, in order to solve the problems of low scheduling efficiency and low resource utilization of heterogeneous storage resources in the existing technology in a cross-cloud environment.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for unified scheduling of heterogeneous storage resources across cloud platforms, comprising: Obtain heterogeneous storage resource information and virtual machine storage requirement information from multiple cloud platforms, wherein the heterogeneous storage resource information includes a tagged dataset; The labeled dataset is digitized to generate a set of feature vectors, and a resource topology map is constructed based on the set of feature vectors. A resource scheduling algorithm is used to perform collaborative analysis on the feature vector set, the resource topology map, and the storage requirement information. Based on the analysis results, the target heterogeneous resources and resource configuration scheme are calculated. Based on the resource configuration scheme, the target heterogeneous resources are virtualized to form a virtual storage pool; A load-aware scheduling method is adopted to uniformly manage and allocate virtualized target heterogeneous resources within the virtual storage pool on demand, thereby achieving unified scheduling of heterogeneous storage resources across cloud platforms.

[0007] Optionally, the step of employing a resource scheduling algorithm to collaboratively analyze the feature vector set, the resource topology map, and the storage requirement information, and calculating the target heterogeneous resources and resource allocation scheme based on the analysis results, includes: The feature vector set and the storage requirement information are compared and similarity is calculated to generate a preliminary matching result; Based on the resource topology map, the preliminary matching results are optimized by topology constraints. Based on the optimized matching results, a set of candidate resources that meet the preset network latency requirements is determined. Based on the geographical distribution constraints in the storage demand information, the candidate resource set is filtered to determine the target heterogeneous resources; Based on the target heterogeneous resources, a resource allocation scheme is generated.

[0008] Optionally, the step of filtering the candidate resource set based on the geographical distribution constraints in the storage demand information to determine the target heterogeneous resources includes: The candidate resource set is filtered using a spatial matching algorithm to obtain a group of alternative resources that meets the geographical distribution constraints. Based on the capacity requirements and performance indicators in the storage requirement information, the candidate resource groups are verified and evaluated to select an effective set of resources that meet all the requirements. The effective resource set is subjected to load balancing analysis using a resource scheduling algorithm to obtain the load weight and performance utilization of each resource node; Based on the load weight and the performance utilization rate, a multi-objective optimization algorithm is used to determine the target heterogeneous resources from the set of effective resources.

[0009] Optionally, the step of using a resource scheduling algorithm to perform load balancing analysis on the effective resource set to obtain the load weight and performance utilization of each resource node includes: Collect real-time operational data corresponding to each resource node in the effective resource set; The real-time operating data is classified and weighted according to resource type to generate target load indicators; The target load index is processed by a dynamic weight allocation algorithm to calculate the load weight of each resource node. Based on the target load index and combined with resource capacity data, the utilization rate index of each resource node is calculated.

[0010] Optionally, the step of digitizing the labeled dataset to generate a set of feature vectors, and constructing a resource topology map based on the set of feature vectors, includes: The multiple storage resources in the labeled dataset are parsed, and the attribute features corresponding to each storage resource are extracted; The attribute features are converted into standardized numerical representations, and a set of feature vectors with a unified format is formed based on the standardized numerical representations corresponding to all the storage resources. Geographic coordinate codes, index quantification values, and protocol feature vectors are extracted from the set of feature vectors. Based on the geographic coordinate encoding, calculate the network distance and transmission latency between each storage resource; Based on the quantified values ​​of the indicators and the protocol feature vectors, establish the connection relationship between storage resources; Based on the network distance, the transmission delay, and the connection relationship, a resource topology map is constructed.

[0011] Optionally, the virtualization of the target heterogeneous resources based on the resource configuration scheme to form a virtual storage pool includes: According to the resource configuration scheme, establish communication connections with the target heterogeneous resources of each cloud platform; Based on the communication connection, the target heterogeneous resources are virtualized and abstracted, and the virtualized and abstracted target heterogeneous resources are converted into virtual storage units of a unified format. According to the resource allocation strategy in the resource configuration scheme, an independent access control strategy and isolation mechanism are configured for each virtual storage unit to obtain an optimized virtual storage unit, and a target data channel is established between the optimized virtual storage units. Configure the corresponding quality of service level and performance guarantee policy for each virtual storage unit with a target data channel; All configured virtual storage units are managed according to a unified namespace to form a virtual storage pool.

[0012] Optionally, the method of using load-aware scheduling to uniformly manage and allocate virtualized target heterogeneous resources within the virtual storage pool includes: Obtain the performance index data of each virtual storage unit in the virtual storage pool; Based on the performance index data, calculate the real-time load weight of each virtual storage unit, and identify the key storage units whose real-time load weight is higher than the preset load threshold. Based on the aforementioned key storage units and combined with storage priority, a dynamic resource adjustment strategy is generated. According to the dynamic resource adjustment strategy, the critical storage units are subjected to load balancing. Based on the distribution results and information on changes in resource demand, the allocated storage resource capacity is dynamically expanded or contracted.

[0013] Secondly, this application provides a unified scheduling system for heterogeneous storage resources across cloud platforms, comprising: The acquisition module is used to acquire heterogeneous storage resource information and virtual machine storage requirement information from multiple cloud platforms, wherein the heterogeneous storage resource information includes a tagged dataset. The generation module is used to digitize the labeled dataset, generate a set of feature vectors, and construct a resource topology map based on the set of feature vectors. The analysis module is used to perform collaborative analysis on the feature vector set, the resource topology map, and the storage requirement information using a resource scheduling algorithm, and calculate the target heterogeneous resources and resource configuration scheme based on the analysis results; A forming module is used to virtualize the target heterogeneous resources based on the resource configuration scheme to form a virtual storage pool; The allocation module is used to uniformly manage and allocate virtualized target heterogeneous resources in the virtual storage pool on demand using a load-aware scheduling method, so as to complete the unified scheduling of heterogeneous storage resources across cloud platforms.

[0014] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor, used to execute the computer program, implements the steps of the unified scheduling method for heterogeneous storage resources across cloud platforms as described in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the unified scheduling method for heterogeneous storage resources across cloud platforms as described in the first aspect above.

[0016] The unified scheduling method for heterogeneous storage resources across cloud platforms provided in this application has the following beneficial effects: This application starts from a comprehensive understanding of the resource status and business needs in a cross-cloud environment, laying a data foundation for intelligent scheduling decisions; on this basis, the underlying heterogeneous storage resources are transformed into a unified and computable data model, thereby clearly presenting the relationships between resources and the network topology; then, through multi-dimensional intelligent matching and optimization calculation, the optimal resource configuration scheme that can simultaneously take into account factors such as performance, geographical location, and cost is finally generated; this scheme can also effectively shield the technical differences of the underlying infrastructure and provide standardized storage service interfaces to the outside world; and through the dynamic adjustment and elastic scaling mechanism of resources, the balance between resource supply and business demand is continuously maintained.

[0017] Furthermore, this application improves the accuracy of resource matching and scheduling efficiency through a multi-level optimization process from initial screening to precise positioning, thereby effectively ensuring the scientific and rational allocation of storage resources in a cross-cloud environment.

[0018] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for unified scheduling of heterogeneous storage resources across cloud platforms, provided in an embodiment of this application;

[0021] Figure 2 A schematic diagram illustrating a specific implementation of a unified scheduling method for heterogeneous storage resources across cloud platforms, provided in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the structure of a unified scheduling system for heterogeneous storage resources across cloud platforms, provided in an embodiment of this application. Detailed Implementation

[0023] In the field of heterogeneous storage resource scheduling across cloud platforms, the current mainstream solution based on resource tag matching has significant limitations in practical applications. This is mainly because the solution relies on preset static tags for resource screening and lacks the ability to dynamically perceive and respond to changes in cloud network topology and real-time load status. As a result, data transmission bottlenecks or performance fluctuations can easily occur during resource selection. At the same time, the fixed weight allocation mechanism it adopts is also difficult to flexibly adapt to the changing application scenario requirements, resulting in unsatisfactory performance in resource utilization optimization. In addition, the ability to guarantee performance isolation between storage resources is relatively limited, which may further have a potential impact on the stability of critical business operations.

[0024] To address the limitations of existing solutions, this application proposes a unified scheduling method for heterogeneous storage resources across cloud platforms. The core idea of ​​this method is as follows: First, a set of feature vectors and a resource topology map are constructed through digital processing, thereby achieving unified modeling of heterogeneous storage resources and a visual representation of their relationships. Based on this, the resource scheduling algorithm performs collaborative analysis of the feature vectors, resource topology map, and specific storage requirements, and generates an optimal resource configuration scheme that takes into account various constraints through multi-level optimization calculations. Finally, this method leverages virtualization technology and a load-aware scheduling mechanism to achieve unified management and on-demand elastic allocation of heterogeneous resources.

[0025] Therefore, this method can dynamically sense changes in resource status and network topology, which significantly improves the accuracy of resource selection and the adaptability of scheduling strategies. This effectively solves the shortcomings of existing solutions in terms of transmission bottlenecks, performance fluctuations and resource utilization, thereby improving the overall efficiency and operational reliability of cross-cloud storage resource scheduling.

[0026] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The core of this application is to provide a unified scheduling method for heterogeneous storage resources across cloud platforms, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes: Step 101: Obtain heterogeneous storage resource information and virtual machine storage requirement information from multiple cloud platforms. The heterogeneous storage resource information includes a tagged dataset.

[0028] In step 101, a cloud platform refers to the software and hardware infrastructure environment that provides cloud computing services and can provide various computing resources and services on demand; heterogeneous storage resource information refers to storage resource data with different characteristics and interfaces provided by different cloud platforms. For example, the heterogeneous storage resource information includes information such as storage type, capacity, performance indicators and geographical location; the tagged dataset is a data set formed by marking and organizing these resource information in a standardized format, which is convenient for subsequent unified processing.

[0029] A virtual machine is a virtual computer instance created on a cloud platform using virtualization technology. It runs on a physical server but has an independent operating system and application environment. The storage requirement information of a virtual machine refers to the specific requirements of a virtual machine running on a cloud platform for storage resources. This storage requirement information includes information such as the required storage type, capacity, performance requirements, and geographical location preferences.

[0030] For example, we obtain block storage resource information from cloud platform A, which has a capacity of 100TB, a performance level of 10,000 read / write operations per second, and is located in North China; we obtain object storage resource information from cloud platform B, which has a capacity of 200TB, a throughput of 500MB per second, and is located in East China; and we obtain file storage resource information from cloud platform C, which has a capacity of 150TB, a performance level of 8,000 read / write operations per second, and is located in South China. This storage resource information from different cloud platforms is standardized and labeled according to multiple dimensions such as storage type, capacity, performance, and geographical location, thus forming a unified labeled dataset. Simultaneously, we obtain the storage requirements information of the virtual machine to be deployed, which specifies that it needs block storage with a capacity of 50TB, a performance requirement of over 8,000 read / write operations per second, and a geographical location in North China.

[0031] Step 102: Digitize the labeled dataset to generate a set of feature vectors, and construct a resource topology map based on the set of feature vectors.

[0032] In step 102, the resource topology graph is a graph structure that represents the network connection relationship and geographical distribution between storage resources of multiple cloud platforms, where nodes represent storage resources and edges represent network connections between resources.

[0033] In this embodiment, firstly, features are extracted from each storage resource in the tagged dataset, converting textual tags into numerical features. Specifically, this includes mapping storage types to numbers through type encoding, directly using quantized values ​​for performance metrics, representing geographical locations using latitude and longitude coordinates, and converting protocol support through feature vector encoding. Then, all converted storage resource features are combined in a unified format to form a complete feature vector set. Based on this, the network distance and transmission delay between each resource node are calculated based on the geographical location information in the feature vector set, and the connection weights between resource nodes are further established by combining performance metrics, thereby ultimately constructing a topology map that reflects the resource association relationships in the entire cross-cloud environment.

[0034] For example, the tagged dataset obtained in step 101 is digitized, such as encoding block storage type as the number 1, object storage as the number 2, and file storage as the number 3; performance indicators are directly represented by raw numerical values; geographical location is described by latitude and longitude coordinates; protocol support is expressed in three-dimensional vector encoding form; after encoding, the feature vector corresponding to the block storage resources of cloud platform A is [1, 10000, a1, a2, 1, 0, 0], the feature vector of the object storage resources of cloud platform B is [2, 500, b1, b2, 0, 1, 0], and the feature vector of the file storage resources of cloud platform C is [3, 8000, c1, c2, 0, 0, 1].

[0035] Based on the geographic coordinates in the feature vectors above, the geographic distance between different resource nodes can be calculated using the distance formula between two points. The expression of this distance formula can be found in relevant technologies and will not be elaborated here. For example, the distance between node A and node B is 1064 kilometers, and the corresponding network transmission delay is approximately 10.64 milliseconds. Finally, based on the distance, delay, and resource characteristics calculated above, a complete resource topology map containing three nodes, A, B, and C, and their interconnections is constructed.

[0036] It should be understood that the specific implementation process of step 102 can be as described in steps 501 to 506 below, and will not be repeated here.

[0037] Step 103: Use a resource scheduling algorithm to perform collaborative analysis on the feature vector set, the resource topology map, and the storage requirement information. Based on the analysis results, calculate the target heterogeneous resources and resource configuration scheme.

[0038] In step 103, the target heterogeneous resources are the set of storage resources that best meet the requirements, calculated by an algorithm. The resource configuration scheme is a detailed configuration and connection scheme for the selected resources.

[0039] In this embodiment, firstly, the feature vector set and storage requirement information are matched for similarity to generate preliminary resource matching results. Then, based on the constructed resource topology map, the preliminary results are optimized and filtered according to the network latency dimension to obtain a set of candidate resources that meet specific latency requirements. Next, based on the geographical distribution constraints specified in the storage requirements, the candidate resource set is further filtered by geographical location. Finally, the load weight and performance utilization of each candidate resource are calculated using a multi-objective optimization algorithm. From the qualified resources that meet all the aforementioned conditions, the optimal target heterogeneous storage resource is selected, and a complete configuration scheme including specific resource allocation, network configuration, and performance guarantee strategies is generated.

[0040] For example, the cosine similarity formula is used to calculate the similarity between the feature vector obtained in step 102 and the virtual machine storage requirements. For instance, the calculation results show that the similarity between the block storage resources of cloud platform A and the virtual machine storage requirements is 0.92. Then, the network latency optimization screening of the resource is performed based on the resource topology map, requiring the latency to be less than 30 milliseconds, while the latency of node A is 20 milliseconds, which meets the requirement.

[0041] Next, based on the geographical location constraint of North China, node A, being located in North China, also meets the requirements. Subsequently, the load weight of node A is calculated to be 60%, and the performance utilization rate is 80%. Combining the above multi-dimensional screening and calculation results, the block storage resources of cloud platform A are finally determined as the target heterogeneous resources, and a complete configuration plan is generated. This plan includes allocating 50TB of storage capacity to the virtual machine, configuring the corresponding network connection, and setting up a continuous performance monitoring strategy.

[0042] It should be understood that the specific implementation process of step 103 can be as described in steps 201 to 204 below, and will not be repeated here.

[0043] Step 104: Based on the resource configuration scheme, the target heterogeneous resources are virtualized to form a virtual storage pool.

[0044] In step 104, the virtual storage pool is a collection of logical storage resources formed by integrating virtualized storage resources.

[0045] In this embodiment, a communication connection is established with the target storage resource according to the generated configuration scheme, and the heterogeneous storage resources of different cloud platforms are converted into virtual storage units of a unified format using virtualization technology. Then, according to the specific requirements of the configuration scheme, corresponding access control policies and resource isolation mechanisms are set for each virtual storage unit. On this basis, a high-speed data channel is established between each virtual storage unit, and clear service quality levels and performance guarantee policies are configured. Finally, all virtual storage units are organized and managed according to a unified namespace, thereby logically forming a single and centralized virtual storage pool.

[0046] For example, establish a secure connection with block storage resources in Cloud Platform A, and convert 100TB of physical storage resources into virtual block storage units of a unified format through block device virtualization technology; then, set access permissions for the virtual block storage unit according to the configuration scheme, that is, the administrator account has read and write permissions, and ordinary user accounts only have read-only permissions; at the same time, set a performance isolation policy to ensure that its minimum performance is not lower than the guaranteed value of 8000 read and write operations per second;

[0047] Based on this, a high-speed data channel with a transmission rate of 10Gbps is established and configured with gold-level service quality, specifically including a guaranteed bandwidth of no less than 8Gbps and a latency of no more than 0.5 milliseconds. Finally, these configured virtual storage units are integrated to form a virtual storage pool called "video-storage". The total available capacity of this storage pool is 50TB, and it has a performance guarantee of no less than 8,000 read and write operations per second.

[0048] Step 105: Use a load-aware scheduling method to uniformly manage and allocate the virtualized target heterogeneous resources in the virtual storage pool on demand, so as to complete the unified scheduling of heterogeneous storage resources across cloud platforms.

[0049] In step 105, unified management refers to centralized monitoring and scheduling of all resources in the virtual storage pool; on-demand allocation refers to dynamically allocating and adjusting storage resource capacity according to actual needs.

[0050] In this embodiment, firstly, the performance metrics of various storage resources in the virtual storage pool are monitored in real time. These performance metrics may include read / write speed, bandwidth utilization, and latency. Then, based on the acquired real-time monitoring data, the real-time load weight of each resource is calculated to identify the currently high-load resources. Next, combined with predefined storage resource priorities, a corresponding dynamic adjustment strategy is formulated to perform load balancing on the identified high-load resources. Finally, based on the load balancing results and changes in actual storage demand, the allocated storage resource capacity is dynamically expanded or contracted to continuously maintain a dynamic balance between resource supply and business demand.

[0051] For example, monitoring revealed that the current virtual storage pool was experiencing 9,000 read / write operations per minute, with a bandwidth utilization of 80% and a latency of 5 milliseconds. If the current load weight is 0.79, this value exceeds the preset threshold of 0.75. Based on this, a traffic distribution strategy was implemented, migrating 20% ​​of the total load, or 1,800 read / write operations per minute, to backup resource nodes. Simultaneously, based on a 30% increase in current business demand, the storage capacity of the virtual storage pool was dynamically expanded from 50TB to 65TB, and the corresponding performance guarantee was improved from 8,000 read / write operations per minute to 9,000 read / write operations per minute.

[0052] This method achieves intelligent unified scheduling of heterogeneous storage resources across cloud platforms through multi-dimensional resource feature analysis and dynamic scheduling optimization, thereby providing stable and efficient storage service support for cross-cloud applications.

[0053] To address the issues of insufficient matching accuracy and unreasonable resource allocation in cross-cloud platform storage resource scheduling, some embodiments, such as Figure 2 As shown, step 103: A resource scheduling algorithm is used to perform collaborative analysis on the feature vector set, the resource topology map, and the storage requirement information. Based on the analysis results, the target heterogeneous resources and resource configuration scheme are calculated, including:

[0054] Step 201: Perform similarity matching calculation between the feature vector set and the storage requirement information to generate preliminary matching results.

[0055] In step 201, the preliminary matching result is a sorted list of matching degrees between resources and requirements obtained after preliminary calculation.

[0056] In this embodiment, the similarity between each resource feature vector in the feature vector set and the demand feature vector in the storage demand information is calculated. The cosine similarity between the two is calculated using a vector space model, thereby obtaining a matching score between each storage resource and the storage demand. Then, all storage resources are sorted from high to low according to the calculated matching scores, and a preliminary matching result list is generated accordingly for use in subsequent steps.

[0057] Step 202: Based on the resource topology map, perform topology constraint optimization on the preliminary matching results, and determine the candidate resource set that meets the preset network latency requirements based on the optimized matching results.

[0058] In step 202, network latency requirements may include: the maximum acceptable network transmission latency threshold for the application, the real-time requirements for data transmission, and specific constraints on network latency for different service types; the candidate resource set, i.e., the above analysis results, is a set of resources that meet network performance requirements after topology optimization.

[0059] In this embodiment of the application, based on the network connection relationship and latency data between resource nodes recorded in the resource topology map, network latency verification is performed on each storage resource in the preliminary matching result to calculate the transmission latency between each resource node and the demand node; then, storage resources with calculated latency values ​​less than a preset threshold are selected; finally, a set of candidate resources that meet the network latency constraints is formed.

[0060] Step 203: Based on the geographical distribution constraints in the storage demand information, filter the candidate resource set to determine the target heterogeneous resources.

[0061] In step 203, the geographical distribution constraints in the storage demand information are derived from the storage deployment geographical location requirements proposed by the user or application. This means that there are specific restrictions on the physical location of the storage resources, such as requiring the storage resources to be located in a specific region or data center.

[0062] In this embodiment of the application, based on the geographical distribution constraints specified in the storage requirement information, a geographical location compliance check is performed on each storage resource in the candidate resource set. Specifically, the geographical location of the resource is verified to meet the geographical area requirements specified in the storage requirements, thereby ultimately determining the storage resource that meets the geographical requirements as the target heterogeneous resource.

[0063] Step 204: Generate a resource configuration scheme based on the target heterogeneous resources.

[0064] In this embodiment of the application, a complete configuration scheme is formulated based on the specific characteristics of the identified target heterogeneous resources and the detailed requirements in the storage demand information. The scheme specifically includes resource capacity allocation, network connection configuration, performance parameter settings and security policies, etc., to ensure that the configured resources can fully meet the actual usage needs of the virtual machine.

[0065] Here is a specific example: Following the aforementioned video processing business scenario, this embodiment first performs similarity matching calculation between the feature vector set and the storage requirement information. The requirement vector corresponding to the storage requirement information is represented as block storage type code 1, performance requirement of 8000 read / write operations per second, and geographical location of North China, with latitude and longitude of a1 and a2. The similarity is calculated using the cosine similarity formula. It should be understood that the above latitude and longitude values ​​and other related values ​​given later are virtual and hypothetical geographical information, not real geographical information.

[0066] In the specific calculation, O = (A·B) / (||A||×||B||), where O represents the similarity, A is the resource feature vector, and B is the demand feature vector. If a1 is 116.4074, a2 is 39.9042, and the feature vector of the block storage resource of cloud platform A is [1, 10000, 116.4074, 39.9042, 1, 0, 0], and the demand vector is [1, 8000, 116.4074, 39.9042, 1, 0, 0], then the dot product result is... The magnitude of vector A is The magnitude of vector B is ,but In the preliminary matching results list generated as a result, the block storage resource of cloud platform A ranked first with a similarity of 0.9998;

[0067] Next, the preliminary matching results were optimized based on the resource topology map. The map showed that the network latency from the A cloud platform node to the demand node was 20 milliseconds, which met the preset network latency requirement of 30 milliseconds. Thus, it was determined that the candidate resource set included the block storage resources of the A cloud platform. Then, based on the geographical distribution constraint of North China in the storage demand information, the candidate resource set was screened. Since the A cloud platform is located in North China, it meets the geographical requirement and is therefore identified as the target heterogeneous resource.

[0068] Finally, a specific resource configuration plan is generated based on the identified target heterogeneous resources. This plan includes allocating 50TB of capacity, configuring a dedicated network connection channel, setting the performance monitoring threshold to 8,000 read / write operations per second, and deploying a data encryption transmission mechanism to ensure that the configured storage resources can fully meet the various needs of the video processing business.

[0069] In this embodiment, a multi-level screening and optimization process is used to achieve a precise match between storage resources and demand, thereby providing reliable storage assurance for cross-cloud applications.

[0070] To further improve the accuracy of screening and the rationality of resource selection, in some embodiments, step 203: based on the geographical distribution constraints in the storage demand information, the candidate resource set is screened to determine the target heterogeneous resources, including:

[0071] Step 301: Use a spatial matching algorithm to filter the candidate resource set to obtain alternative resource groups that meet the geographical distribution constraints.

[0072] In step 301, the candidate resource group is a set of resources that meet the geographical distribution constraints after spatial matching and filtering.

[0073] In this embodiment of the application, a spatial matching algorithm is used to calculate the spatial relationship between the geographic coordinates of each resource node in the candidate resource set and the geographic area range specified in the storage requirements. Then, by determining whether the coordinates of the resource node fall within the polygon range of the specified geographic area, it is specifically determined whether the resource meets the preset geographic constraints. Finally, all resource nodes that meet the conditions are combined into a candidate resource group.

[0074] It should be noted that the embodiments of this application do not impose specific limitations on the algorithm type and specific implementation process of the spatial matching algorithm, and can be set accordingly according to the actual situation.

[0075] Step 302: Based on the capacity requirements and performance indicators in the storage requirement information, verify and evaluate the candidate resource group to select an effective resource set that meets all the requirements.

[0076] In step 302, the capacity requirements and performance indicators in the storage requirement information are derived from the storage resource configuration requirements of the virtual machine or application. The capacity requirement refers to the required storage space size, and the performance indicator requirements include specific performance parameters such as IOPS, throughput, and access latency. All requirement conditions specifically refer to the set of all constraints that the storage resource configuration must meet, such as capacity requirements, performance indicator requirements, geographical distribution constraints, and network latency requirements. The effective resource set is the resource set that has passed both capacity verification and performance evaluation.

[0077] In this embodiment of the application, the available capacity of each storage resource in the candidate resource group is checked to see if it is greater than or equal to the required capacity according to the capacity requirement value explicitly indicated in the storage requirement information. Then, the actual performance index of each storage resource is tested to see if it reaches the minimum threshold of the required performance requirement. In this way, all storage resources that meet both the capacity requirement and the performance requirement are selected to form the final effective resource set.

[0078] For example: Assume the storage requirement is a capacity of 100GB and a read / write speed of no less than 500MB / s. Available alternative resource A has an available capacity of 200GB and a measured read / write speed of 600MB / s; alternative resource B has an available capacity of 80GB and a measured read / write speed of 700MB / s; and alternative resource C has an available capacity of 150GB and a measured read / write speed of 400MB / s. Verification shows that resource A meets both the capacity and performance requirements. Resource B, with an available capacity of only 80GB, fails to meet the capacity requirement, and resource C's measured read / write speed does not reach the minimum performance threshold. Therefore, only resource A is selected into the final set of valid resources.

[0079] Step 303: Use a resource scheduling algorithm to perform load balancing analysis on the effective resource set to obtain the load weight and performance utilization of each resource node.

[0080] In step 303, each resource node refers to the physical storage device or storage system represented in the resource topology map, specifically the hardware device or storage cluster that actually provides storage services; load weight is a quantitative indicator reflecting the current load level of a resource node, and performance utilization is the ratio of the current performance usage of a resource node to its maximum performance capacity.

[0081] In this embodiment of the application, a resource scheduling algorithm is used to collect real-time operating data of each resource node in the effective resource set. The real-time operating data includes key performance indicators such as processing capacity utilization, memory utilization, and storage bandwidth utilization. Then, the real-time load weight of each resource node is obtained through weighted calculation, and the utilization rate index of each resource node is calculated at the same time, thereby providing necessary data support for the subsequent optimization and selection process.

[0082] Step 304: Based on the load weight and the performance utilization rate, use a multi-objective optimization algorithm to determine the target heterogeneous resources from the effective resource set.

[0083] In this embodiment of the application, based on the acquired load weight and performance utilization data, a multi-objective optimization algorithm is used to establish an optimization objective function that comprehensively considers multiple objectives such as load balancing and performance utilization efficiency. Then, based on this function, the algorithm is used to calculate and select the resource with the best overall performance from the effective resource set, and determine it as the final target heterogeneous resource.

[0084] It should be noted that this embodiment does not impose specific limitations on the expression of the optimization objective function, and can be set accordingly based on the actual situation.

[0085] In this embodiment, multi-level refined screening and optimization decisions ensure that resource selection meets geographical constraints, thereby providing more reliable storage guarantees for business operations.

[0086] To further improve the accuracy and practicality of load balancing analysis, in some embodiments, step 303: using a resource scheduling algorithm to perform load balancing analysis on the effective resource set to obtain the load weight and performance utilization of each resource node, including:

[0087] Step 401: Collect real-time running data corresponding to each resource node in the effective resource set.

[0088] Step 402: Classify and weight the real-time operating data according to resource type to generate target load indicators.

[0089] In step 402, the resource type refers to the classification category of storage resources, which means the types of resources divided according to different storage characteristics and service forms. These resource types include block storage resources, file storage resources, and object storage resources. The target load index is the comprehensive load value obtained by weighting the operating data of different types of resources according to a unified standard.

[0090] In this embodiment of the application, the collected real-time operation data is classified and processed according to resource type, and a corresponding weight coefficient is assigned to each resource type. Then, the utilization rate data of various resources are merged into a unified target load index through a weighted calculation formula, thereby eliminating the comparison differences between different types of resources due to different units of measurement.

[0091] Step 403: Process the target load index using a dynamic weight allocation algorithm to calculate the load weight of each resource node.

[0092] In this embodiment, the characteristics of the current operating state are analyzed by a dynamic weight allocation algorithm, and dynamic weight coefficients are assigned to the target load index. Then, the real-time load weight of each resource node is calculated in combination with the specific situation of the real-time load. The weight value can accurately reflect the current comprehensive load pressure of each node.

[0093] It should be noted that this embodiment does not impose specific limitations on the expression and algorithm type of the dynamic weight allocation algorithm, and can be set accordingly based on the actual situation.

[0094] Step 404: Based on the target load index and combined with the resource capacity data, calculate the utilization rate index of each resource node.

[0095] In step 404, the resource capacity data comes from the actual configuration information of the storage resources of each cloud platform. It means the total amount of physical storage space provided by the storage device or storage system. The total amount of physical storage space includes specific values ​​such as total capacity, used capacity and available capacity.

[0096] In this embodiment of the application, based on the specific value of the target load index and combined with the maximum performance capacity data of each resource node, the utilization index of each resource node is calculated by the utilization calculation formula. The utilization index represents the proportion of the current actual usage of the resource node in its total capacity.

[0097] It should be noted that this embodiment does not impose specific limitations on the expression of the utilization rate calculation formula, and can be set accordingly according to the actual situation.

[0098] In this embodiment, accurate quantitative assessment of resource load status is achieved through multi-dimensional data collection and intelligent calculation analysis, which effectively improves resource utilization efficiency and system operation stability.

[0099] To address the issues of unified representation and relational modeling of heterogeneous storage resources, in some embodiments, step 102 involves: digitizing the labeled dataset to generate a set of feature vectors, and constructing a resource topology map based on the set of feature vectors, including:

[0100] Step 501: parse multiple storage resources in the labeled dataset and extract the attribute features corresponding to each storage resource.

[0101] In step 501, attribute characteristics refer to the inherent characteristic parameters of storage resources. These attribute characteristics include data describing the essential attributes of storage resources, such as storage type characteristics, performance level characteristics, geographical location characteristics, and protocol support characteristics.

[0102] In this embodiment of the application, each storage resource record in the tagged dataset is parsed to extract key attribute features such as storage type, performance indicators, geographic location coordinates and protocol support. These extracted features are then organized into a unified structured data format for use in subsequent processing steps.

[0103] Step 502: Convert the attribute features into standardized numerical representations, and form a set of feature vectors with a unified format based on the standardized numerical representations corresponding to all the storage resources.

[0104] In step 502, the standardized numerical representation is the process of converting different types of attribute features into a uniform numerical format.

[0105] In this embodiment of the application, the extracted attribute features are converted into numerical values. The storage type can be mapped using a preset encoding, the performance indicators can be directly represented by their original values, the geographical location can be represented by latitude and longitude coordinates, and the protocol support can be represented by vector encoding. Then, all the standardized values ​​of each storage resource are combined in a preset fixed order to form the feature vector corresponding to the resource. Finally, the feature vectors of all resources together constitute a complete feature vector set.

[0106] Step 503: Extract the geographic coordinate code, index quantization value and protocol feature vector from the feature vector set.

[0107] In step 503, the geographic coordinate encoding is the part of the feature vector that represents the geographic location, the index quantification value is the part of the feature vector that represents the performance level, and the protocol feature vector is the part of the feature vector that represents the protocol support.

[0108] Step 504: Based on the geographic coordinate encoding, calculate the network distance and transmission delay between each storage resource.

[0109] In step 504, network distance refers to the physical network distance between two storage resource nodes, and transmission delay refers to the time delay required for data to be transmitted between the two nodes.

[0110] In this embodiment, based on latitude and longitude data in geographic coordinate encoding, the network distance between each pair of storage resource nodes can be calculated using the spherical distance formula. Then, the corresponding transmission delay value is estimated based on the calculated distance data, thereby establishing basic data that reflects the network connectivity relationship between nodes. It should be noted that the spherical distance formula can refer to related technologies, and will not be elaborated upon in this embodiment.

[0111] Step 505: Based on the quantified value of the indicator and the protocol feature vector, establish the connection relationship between storage resources.

[0112] In step 505, the connection relationship is an association that reflects the degree of performance matching between storage resources, and is used to represent the performance compatibility and coordination between resources.

[0113] In this embodiment of the application, the performance difference between different storage resources is calculated based on the numerical value of the index, and the protocol compatibility is calculated by combining the protocol feature vector. Then, the connection relationship between storage resources is established by weighted calculation. The higher the weight value, the better the performance matching between the associated resources.

[0114] Step 506: Construct a resource topology map based on the network distance, the transmission delay, and the connection relationship.

[0115] In this embodiment, physical connections between resource nodes are established based on network distance and transmission delay data. The specific weight value of each connection edge is determined in combination with the established connections. Finally, all resource nodes and their connections are integrated to construct a complete resource topology map.

[0116] In this embodiment of the application, a unified representation and visualization of the relationships between heterogeneous storage resources are achieved through systematic digital processing of resource characteristics and topological relationship modeling, thereby providing complete knowledge graph support for subsequent resource scheduling.

[0117] To address the issue of unified management and efficient utilization of heterogeneous storage resources, in some embodiments, step 104: based on the resource configuration scheme, virtualize the target heterogeneous resources to form a virtual storage pool, including:

[0118] Step 601: Establish communication connections with the target heterogeneous resources of each cloud platform according to the resource configuration scheme.

[0119] In step 601, the communication connection refers to a reliable communication link established between the target heterogeneous resources and the scheduling system of each cloud platform through a network security protocol for data transmission and control. This connection ensures secure access to and operation of remote storage resources.

[0120] In this embodiment of the application, based on the target heterogeneous resource access information recorded in the resource configuration scheme, an encrypted connection channel is established with each cloud platform using a secure communication protocol to ensure the security and reliability of data transmission, thereby laying the communication foundation for subsequent virtualization processing.

[0121] Step 602: Based on the communication connection, the target heterogeneous resources are virtualized and abstracted, and the virtualized and abstracted target heterogeneous resources are converted into virtual storage units of a unified format.

[0122] In step 602, virtualization abstraction is a technical process of converting physical storage resources into logical storage units. A virtual storage unit is a standardized logical storage unit formed after virtualization processing.

[0123] In this embodiment of the application, the target heterogeneous resources are virtualized and abstracted based on the established encrypted communication connection. That is, the heterogeneous storage resources from different cloud platforms are converted into virtual storage units of a unified format through virtualization technology, thereby shielding the specific differences of the underlying physical resources and providing a consistent storage service interface to the outside world.

[0124] Step 603: According to the resource allocation strategy in the resource configuration scheme, configure an independent access control strategy and isolation mechanism for each virtual storage unit to obtain an optimized virtual storage unit, and establish a target data channel between the optimized virtual storage units.

[0125] In step 603, the access control policy is a security rule that specifies user access permissions, the isolation mechanism is a technical measure to ensure the performance independence of different virtual storage units, and the target data channel is a high-speed data transmission path between virtual storage units.

[0126] Step 604: Configure the corresponding quality of service level and performance guarantee policy for each virtual storage unit with a target data channel.

[0127] In step 604, the service quality level is a standard that defines the quality level of the storage service, and the performance assurance strategy is a technical solution that ensures that the storage performance meets the predetermined requirements.

[0128] Step 605: Manage all configured virtual storage units according to a unified namespace to form a virtual storage pool.

[0129] In step 605, the unified namespace is a logical naming and management framework provided for all virtual storage units.

[0130] In this embodiment, the standardized encapsulation and centralized management of heterogeneous storage resources are achieved through systematic virtualization processing and unified management, thereby providing reliable and efficient storage support for cross-cloud applications.

[0131] To address the dynamic resource management issue of virtual storage pools, in some embodiments, step 105 involves employing a load-aware scheduling method to uniformly manage and allocate virtualized target heterogeneous resources within the virtual storage pool on demand, including:

[0132] Step 701: Obtain the performance index data of each virtual storage unit in the virtual storage pool.

[0133] In step 701, performance metrics data refers to various metrics that reflect the operating status of virtual storage units. These metrics include real-time monitoring data such as read / write operation rates, bandwidth utilization, and response latency.

[0134] In this embodiment of the application, the performance index data of each virtual storage unit in the virtual storage pool is collected periodically by the monitoring system. The performance index data includes key performance parameters such as the read and write operation frequency, data transmission bandwidth usage and request response time of each storage unit in real time. Then, the collected data is summarized to form a structured monitoring dataset.

[0135] Step 702: Calculate the real-time load weight of each virtual storage unit based on the performance index data, and identify the key storage units whose real-time load weight is higher than the preset load threshold.

[0136] In step 702, the real-time load weight is a quantitative value of the load level calculated by combining multiple performance indicators, and the critical storage unit refers to the virtual storage unit whose load weight exceeds a preset threshold.

[0137] In this embodiment of the application, based on the collected performance index data, a weighted calculation model is used to integrate the current values ​​of various performance indicators to calculate the real-time load weight of each virtual storage unit. Then, the calculated load weight value is compared with a preset load threshold to identify key storage units whose load weight exceeds the threshold.

[0138] Step 703: Based on the key storage units and combined with storage priority, generate a dynamic resource adjustment strategy.

[0139] In step 703, the storage priority is derived from the business importance level configuration in the storage demand information. The storage demand information includes a business criticality identifier, which defines the priority level of different virtual machine storage requests. High priority corresponds to critical business applications, and low priority corresponds to ordinary business applications. The dynamic resource adjustment strategy is a resource allocation scheme formulated for high load conditions.

[0140] In this application embodiment, based on the identified key storage unit information and combined with the different priority levels of storage requirements of each business, a corresponding dynamic resource adjustment strategy is formulated. The strategy specifically includes adjustments such as load balancing ratio, resource expansion scheme, and performance optimization measures.

[0141] Step 704: According to the dynamic resource adjustment strategy, perform load balancing on the critical storage units.

[0142] In this embodiment of the application, according to the load balancing scheme determined in the dynamic resource adjustment strategy, the critical storage unit is subjected to load balancing operation, that is, some read and write operations and data transmission tasks are transferred to storage units with relatively low load through data migration and request redirection technology, thereby reducing the pressure on high-load units.

[0143] Step 705: Based on the distribution results and information on changes in resource demand, dynamically expand or shrink the allocated storage resource capacity.

[0144] In step 705, the information on changes in resource requirements comes from the dynamic adjustment of storage resource requirements during the operation of virtual machines or applications. This means that the requirements for storage capacity, performance indicators, or access modes change in real time due to fluctuations in business load, changes in data processing volume, or changes in application scenarios.

[0145] In this embodiment of the application, the allocated storage resource capacity is dynamically adjusted based on the actual effect of the load balancing operation and the real-time monitoring of resource demand changes. When an increase in demand is detected, resources are expanded, and when a decrease in demand is detected, resources are contracted, thereby continuously maintaining a dynamic balance between resource supply and actual demand.

[0146] In this embodiment, dynamic optimization of storage resource allocation is achieved through real-time monitoring and intelligent scheduling, thereby ensuring the stability and reliability of storage services.

[0147] Figure 3 This application provides a schematic diagram of the structure of a unified scheduling system for heterogeneous storage resources across cloud platforms, and the specific implementation details are as follows: The acquisition module 31 is used to acquire heterogeneous storage resource information and virtual machine storage demand information from multiple cloud platforms, wherein the heterogeneous storage resource information includes a tagged dataset.

[0148] The generation module 32 is used to digitize the labeled dataset, generate a set of feature vectors, and construct a resource topology map based on the set of feature vectors.

[0149] Analysis module 33 is used to perform collaborative analysis on the feature vector set, the resource topology map and the storage demand information using a resource scheduling algorithm, and calculate the target heterogeneous resources and resource configuration scheme based on the analysis results.

[0150] The forming module 34 is used to virtualize the target heterogeneous resources based on the resource configuration scheme to form a virtual storage pool.

[0151] The allocation module 35 is used to uniformly manage and allocate virtualized target heterogeneous resources in the virtual storage pool on demand using a load-aware scheduling method, so as to complete the unified scheduling of heterogeneous storage resources across cloud platforms.

[0152] The cross-cloud platform heterogeneous storage resource unified scheduling system of this application embodiment is used to implement the aforementioned cross-cloud platform heterogeneous storage resource unified scheduling method. Therefore, the specific implementation of the cross-cloud platform heterogeneous storage resource unified scheduling system can be found in the embodiment section of the cross-cloud platform heterogeneous storage resource unified scheduling method above. The specific implementation can be referred to the description of the corresponding embodiment, and will not be repeated here.

[0153] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the cross-cloud platform heterogeneous storage resource unified scheduling method described above.

[0154] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for unified scheduling of heterogeneous storage resources across cloud platforms.

[0155] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0156] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the cross-cloud platform heterogeneous storage resource unified scheduling method.

[0157] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0158] The above provides a detailed description of a unified scheduling method and system for heterogeneous storage resources across cloud platforms provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for unified scheduling of heterogeneous storage resources across cloud platforms, characterized in that, include: Obtain heterogeneous storage resource information and virtual machine storage requirement information from multiple cloud platforms, wherein the heterogeneous storage resource information includes a tagged dataset; The labeled dataset is digitized to generate a set of feature vectors, and a resource topology map is constructed based on the set of feature vectors. A resource scheduling algorithm is used to perform collaborative analysis on the feature vector set, the resource topology map, and the storage requirement information. Based on the analysis results, the target heterogeneous resources and resource configuration scheme are calculated. Based on the resource configuration scheme, the target heterogeneous resources are virtualized to form a virtual storage pool; A load-aware scheduling method is adopted to uniformly manage and allocate virtualized target heterogeneous resources within the virtual storage pool on demand, thereby achieving unified scheduling of heterogeneous storage resources across cloud platforms.

2. The method according to claim 1, characterized in that, The method employs a resource scheduling algorithm to collaboratively analyze the feature vector set, the resource topology map, and the storage requirement information. Based on the analysis results, it calculates the target heterogeneous resources and resource allocation scheme, including: The feature vector set and the storage requirement information are compared and similarity is calculated to generate a preliminary matching result; Based on the resource topology map, the preliminary matching results are optimized by topology constraints. Based on the optimized matching results, a set of candidate resources that meet the preset network latency requirements is determined. Based on the geographical distribution constraints in the storage demand information, the candidate resource set is filtered to determine the target heterogeneous resources; Based on the target heterogeneous resources, a resource allocation scheme is generated.

3. The method according to claim 2, characterized in that, The step of filtering the candidate resource set based on the geographical distribution constraints in the storage demand information to determine the target heterogeneous resources includes: The candidate resource set is filtered using a spatial matching algorithm to obtain a group of alternative resources that meets the geographical distribution constraints. Based on the capacity requirements and performance indicators in the storage requirement information, the candidate resource groups are verified and evaluated to select an effective set of resources that meet all the requirements. The effective resource set is subjected to load balancing analysis using a resource scheduling algorithm to obtain the load weight and performance utilization of each resource node; Based on the load weight and the performance utilization rate, a multi-objective optimization algorithm is used to determine the target heterogeneous resources from the set of effective resources.

4. The method according to claim 3, characterized in that, The process of using a resource scheduling algorithm to perform load balancing analysis on the effective resource set to obtain the load weight and performance utilization of each resource node includes: Collect real-time operational data corresponding to each resource node in the effective resource set; The real-time operating data is classified and weighted according to resource type to generate target load indicators; The target load index is processed by a dynamic weight allocation algorithm to calculate the load weight of each resource node. Based on the target load index and combined with resource capacity data, the utilization rate index of each resource node is calculated.

5. The method according to claim 1, characterized in that, The step of digitizing the labeled dataset to generate a feature vector set, and constructing a resource topology map based on the feature vector set, includes: The multiple storage resources in the labeled dataset are parsed, and the attribute features corresponding to each storage resource are extracted; The attribute features are converted into standardized numerical representations, and a set of feature vectors with a unified format is formed based on the standardized numerical representations corresponding to all the storage resources. Geographic coordinate codes, index quantification values, and protocol feature vectors are extracted from the set of feature vectors. Based on the geographic coordinate encoding, calculate the network distance and transmission latency between each storage resource; Based on the quantified values ​​of the indicators and the protocol feature vectors, establish the connection relationship between storage resources; Based on the network distance, the transmission delay, and the connection relationship, a resource topology map is constructed.

6. The method according to claim 1, characterized in that, The virtualization process based on the resource configuration scheme to form a virtual storage pool for the target heterogeneous resources includes: According to the resource configuration scheme, establish communication connections with the target heterogeneous resources of each cloud platform; Based on the communication connection, the target heterogeneous resources are virtualized and abstracted, and the virtualized and abstracted target heterogeneous resources are converted into virtual storage units of a unified format. According to the resource allocation strategy in the resource configuration scheme, an independent access control strategy and isolation mechanism are configured for each virtual storage unit to obtain an optimized virtual storage unit, and a target data channel is established between the optimized virtual storage units. Configure the corresponding quality of service level and performance guarantee policy for each virtual storage unit with a target data channel; All configured virtual storage units are managed according to a unified namespace to form a virtual storage pool.

7. The method according to claim 1, characterized in that, The method of using load-aware scheduling to uniformly manage and allocate virtualized heterogeneous resources within the virtual storage pool on demand includes: Obtain the performance index data of each virtual storage unit in the virtual storage pool; Based on the performance index data, calculate the real-time load weight of each virtual storage unit, and identify the key storage units whose real-time load weight is higher than the preset load threshold. Based on the aforementioned key storage units and combined with storage priority, a dynamic resource adjustment strategy is generated. According to the dynamic resource adjustment strategy, the critical storage units are subjected to load balancing. Based on the distribution results and information on changes in resource demand, the allocated storage resource capacity is dynamically expanded or contracted.

8. A unified scheduling system for heterogeneous storage resources across cloud platforms, characterized in that, include: The acquisition module is used to acquire heterogeneous storage resource information and virtual machine storage requirement information from multiple cloud platforms, wherein the heterogeneous storage resource information includes a tagged dataset. The generation module is used to digitize the labeled dataset, generate a set of feature vectors, and construct a resource topology map based on the set of feature vectors. The analysis module is used to perform collaborative analysis on the feature vector set, the resource topology map, and the storage requirement information using a resource scheduling algorithm, and calculate the target heterogeneous resources and resource configuration scheme based on the analysis results; A forming module is used to virtualize the target heterogeneous resources based on the resource configuration scheme to form a virtual storage pool; The allocation module is used to uniformly manage and allocate virtualized target heterogeneous resources in the virtual storage pool on demand using a load-aware scheduling method, so as to complete the unified scheduling of heterogeneous storage resources across cloud platforms.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the cross-cloud platform heterogeneous storage resource unified scheduling method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the cross-cloud platform heterogeneous storage resource unified scheduling method as described in any one of claims 1 to 7.

Images