Dynamic automatic grouping method and system for heterogeneous resource instances under cloudy environment

Abstract

The application discloses a dynamic automatic grouping method and system for heterogeneous resource instances in a multi-cloud environment, comprising: configuring automatic grouping rules for instance groups, obtaining attribute information and topology relationship information of different types of heterogeneous resource instances from multiple cloud platforms; matching the automatic grouping rules according to the attribute information of the heterogeneous resource instances to be grouped, generating a grouping operation containing a topology snapshot, and propagating the grouping operation and the topology change operation as a conflict-free replication data type operation among multiple management nodes, wherein the conflict-free replication data type refers to a distributed data type satisfying the commutative law, associative law and idempotent law; and merging the received conflict-free replication data type operations at each management node to dynamically update the membership relationship of the instance groups. The application can be asynchronously propagated among multiple management nodes, and consistent merging results can be obtained regardless of the propagation order and repeated reception, without the need to perform additional rollback logic, thereby eliminating single point of failure and transaction coordination bottlenecks.

Description

Technical Field

[0001] This application relates to the field of multi-cloud environments, specifically to a method and system for dynamic automatic grouping of heterogeneous resource instances in a multi-cloud environment. Background Technology

[0002] In multi-cloud resource management platforms, operations and maintenance personnel often configure automatic grouping rules to group eligible resource instances into designated instance groups for unified monitoring and policy distribution. Existing automatic grouping solutions typically employ a centralized processing flow, directly executing instance group addition and deletion operations in the central database after matching rules. In scenarios with frequent resource topology changes, there is a time window between grouping operations and topology queries. The read topology view may have expired before being written, leading to instances with changed relationships being incorrectly grouped into instance groups or creating a partially grouped, dirty intermediate state. While distributed transactions can guarantee atomicity to some extent, compensation operations may fail due to network or service failures, potentially leaving the system in an inconsistent state. Furthermore, the transaction coordinator becomes a performance bottleneck and a single point of failure. Summary of the Invention

[0003] To address the shortcomings of existing technologies in practical applications, this application provides, on the one hand, a method for dynamically and automatically grouping heterogeneous resource instances in a multi-cloud environment. This method unifies grouping intent and topology change events into a conflict-free replication data type operation, utilizing the commutative, associative, and idempotent laws satisfied by this data type. On the other hand, this application provides a system for dynamically and automatically grouping heterogeneous resource instances in a multi-cloud environment.

[0004] The first aspect of this application provides a method for dynamic automatic grouping of heterogeneous resource instances in a multi-cloud environment, the method comprising: Configure automatic grouping rules for instance groups to obtain attribute information and topology information of different types of heterogeneous resource instances from multiple cloud platforms; Based on the attribute information of the heterogeneous resource instances to be grouped, the automatic grouping rules are matched to generate a grouping operation that includes a topology snapshot. The grouping operation and the topology change operation are propagated among multiple management nodes as conflict-free replication data type operations. Conflict-free replication data type refers to a distributed data type that satisfies the commutative law, associative law and idempotency law. At each management node, merge the received conflict-free replication data type operations and dynamically update the membership relationship of the instance group based on the merge result.

[0005] Optionally, a grouping operation that generates a topology snapshot may be included, including: Get the topology snapshot corresponding to the heterogeneous resource instances to be grouped; Generate the grouping and adding operation for the heterogeneous resource instance to be grouped; The grouping and addition operation is performed based on the topology snapshot to generate associated sub-instances of heterogeneous resource instances to be grouped.

[0006] Optionally, obtain a topology snapshot corresponding to the heterogeneous resource instances to be grouped, including: Query the globally monotonically increasing version number of the current directed acyclic graph; Extract a list of associated nodes that have a master-slave or cluster relationship with the heterogeneous resource instances to be grouped; Generate a topology snapshot that includes the version number and a list of associated nodes.

[0007] Optionally, merge the received conflict-free replication data type operations at each management node, including: Receive conflict-free replication data type operations from other management nodes and record them in the local operation log; Merge operations in the local operation log based on the operation type and the version number carried. The grouping status of each heterogeneous resource instance is determined based on the merged set of operations.

[0008] Optionally, operations in the local operation log can be merged based on the operation type and the version number carried, including: When both grouping and topology deletion operations exist on the same heterogeneous resource instance, compare the snapshot version number carried by the grouping and topology deletion operation with the topology version number carried by the topology deletion operation. If the topology version number of the topology deletion operation is greater than the snapshot version number of the grouping and adding operation, then the result of the topology deletion operation is retained. If the topology version number of the topology deletion operation is less than or equal to the snapshot version number of the grouping addition operation, then the result of the grouping addition operation is retained.

[0009] Optionally, obtain attribute information and topology information of different types of heterogeneous resource instances from multiple cloud platforms, including: Obtain full or incremental heterogeneous resource instance data from resource management services of multiple cloud platforms; Instance identifiers are generated using corresponding identification strategies for different types of heterogeneous resource instances. Maintain a topological directed acyclic graph that reflects the master-slave or cluster relationships between different types of heterogeneous resource instances.

[0010] Optionally, the membership relationships of the instance group can be dynamically updated based on the merge results, including: Check if the heterogeneous resource instance to be added already exists in the instance group; If the heterogeneous resource instance to be added does not exist in the instance group, then add the instance to the instance group; If the heterogeneous resource instance to be added already exists in the instance group, then refresh the cache status of that instance.

[0011] Optionally, after dynamically updating the membership of the instance group based on the merge result, it also includes: Write the association between heterogeneous resource instances and instance groups into the document database by updating if they exist and inserting if they do not. Write the association index between instances and instance groups to the cache database based on the type differences of heterogeneous resource instances; Record log information about the automatic grouping operation of heterogeneous resource instances.

[0012] Optional, also includes: When an instance group is deleted, an instance group deletion operation is generated and propagated as a conflict-free replication data type operation among multiple management nodes. After each management node merges the instance group deletion operation, remove the associated index of the corresponding heterogeneous resource instance from the cache database; Delete the documents and heterogeneous resource instance relationship documents corresponding to the instance group in the document database; update the alarm policy bound to the instance group.

[0013] This application also discloses a dynamic automatic grouping system for heterogeneous resource instances in a multi-cloud environment, the system comprising: The rules management module is used to configure automatic grouping rules for instance groups and obtain attribute information and topology information of different types of heterogeneous resource instances from multiple cloud platforms. The synchronous propagation module is used to match automatic grouping rules based on the attribute information of the heterogeneous resource instances to be grouped, generate grouping operations that include topology snapshots, and propagate the grouping operations and topology change operations as conflict-free replication data type operations among multiple management nodes. Conflict-free replication data type refers to a distributed data type that satisfies the commutative law, associative law, and idempotency law. The merge update module is used to merge the received conflict-free replication data type operations at each management node and dynamically update the membership relationship of the instance group based on the merge result.

[0014] As can be seen from the above technical solution, this application abstracts the grouping intent and topology change event into a conflict-free replication data type operation. Utilizing the commutative, associative, and idempotent laws satisfied by this data type, the operation can be asynchronously propagated among multiple management nodes, and consistent merging results can be obtained regardless of the propagation order or repeated reception. Simultaneously, this application models the topology relationship as a directed acyclic graph with version numbers. The grouping operation carries a topology snapshot and version number. When the topology changes, the grouping operation generated based on the old snapshot is automatically overwritten by the topology change operation carrying a higher version number, without requiring additional rollback logic. Each management node independently performs operation merging and state updates, without relying on a central coordinator, eliminating single points of failure and transaction coordination bottlenecks. All nodes ultimately reach consensus on the members of the instance group; the state convergence process is lock-free and transaction rollback-free, and the convergence result conforms to the correct expectation after the topology change. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application; Figure 2 This is a flowchart illustrating the generation of a grouping operation including a topology snapshot, in a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 3 This is a flowchart illustrating the process of obtaining the topology snapshot corresponding to the heterogeneous resource instance to be grouped in a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 4 A flowchart illustrating the process of merging conflict-free replicated data types received at each management node in a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 5 This is a schematic diagram illustrating the process of merging operations in the local operation log based on the operation type and the version number carried, according to a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 6This is a flowchart illustrating the process of obtaining attribute information and topology information of heterogeneous resource instances of different types from multiple cloud platforms in a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 7 This is a flowchart illustrating the process of dynamically updating the membership relationship of instance groups based on the merging result in a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 8 This is a flowchart illustrating the process of dynamically updating the membership relationship of instance groups based on the merging results in a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 9 A flowchart illustrating the remaining steps of a specific embodiment of the dynamic automatic grouping method for heterogeneous resource instances in a multi-cloud environment provided in this application. Figure 10 This is a schematic diagram of a structure suitable for implementing the system of this application embodiment. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] This application provides a method for dynamic automatic grouping of heterogeneous resource instances in a multi-cloud environment. This method can be applied to a multi-cloud resource management platform to automatically group and manage different types of resource instances distributed across multiple cloud platforms. The problem that existing technologies need to solve is how to ensure eventual consistency of instance group states under the requirement of multiple management nodes concurrently performing automatic grouping and high-frequency changes in resource topology relationships in a multi-cloud environment, while avoiding compensation failures and performance bottlenecks caused by distributed transactions.

[0019] This application discloses a method for dynamic automatic grouping of heterogeneous resource instances in a multi-cloud environment, such as... Figure 1 As shown, the method includes: S100: Configure automatic grouping rules for instance groups to obtain attribute information and topology information of heterogeneous resource instances of different types from multiple cloud platforms.

[0020] S200: Matches automatic grouping rules based on the attribute information of heterogeneous resource instances to be grouped, generates a grouping operation that includes a topology snapshot, and propagates the grouping operation and topology change operation as conflict-free replication data type operations among multiple management nodes. Conflict-free replication data type refers to a distributed data type that satisfies the commutative, associative, and idempotent laws.

[0021] S300: Merges the received conflict-free replication data type operations at each management node and dynamically updates the membership relationship of the instance group based on the merge result.

[0022] In this embodiment, configuring automatic grouping rules for instance groups can be completed through the user interface of the resource management platform. Operations personnel can find the automatic grouping rule configuration area on the instance group creation or editing page, select different condition dimensions according to actual management needs, and set corresponding attribute values. Automatic grouping rules support combinations of multiple condition dimensions, each corresponding to the attribute characteristics of different types of resource instances. The basic affiliation dimension may include a cloud region identifier list, a dedicated zone identifier list, and a main account identifier list, used to limit the geographical distribution and account to which the resource instance belongs. The device specification dimension may include a device type list, used to limit resource instances with different hardware specifications such as physical servers and graphics processor servers. The database characteristic dimension may include a data source type identifier list and a data source version identifier list, used to limit database resource instances of different types and versions. The storage characteristic dimension may include a cloud disk type list and a cloud disk attribute list, used to limit different types of storage resource instances such as block storage and object storage. The cluster characteristic dimension may include a cluster identifier list and a storage pool name list, used to limit cluster-type resource instances such as distributed storage. The site usage dimension may include a site identifier list and a usage identifier list, used to limit resource instances in different scenarios such as edge nodes and artificial intelligence training platforms. Each condition dimension can have one or more attribute values. When a condition dimension has no attribute value set, it means that the dimension does not participate in rule matching, meaning that all resource instances meet the conditions for that dimension. After the automatic grouping rules are configured, they will be persistently stored in the document database in the form of documents. Each rule is uniquely associated with the corresponding instance group through the instance group identifier field, and the resource type field limits the types of resource instances to which the rule applies.

[0023] In this embodiment, the process of obtaining attribute information and topology information of heterogeneous resource instances of different types from multiple cloud platforms can be achieved through two methods: timed synchronization or event listening. In the timed synchronization method, the system sends data synchronization requests to the resource management services of each cloud platform at preset time intervals to obtain full or incremental resource instance data. In the event listening method, the system establishes a connection with the event notification service of each cloud platform. When a resource instance is created, modified, or deleted on a cloud platform, the cloud platform actively pushes the corresponding event notification to the system, and the system obtains the changed resource instance data based on the event notification. By combining these two methods, the system can obtain the latest status information of all heterogeneous resource instances in a multi-cloud environment in a timely manner. The obtained resource instance data is parsed and processed to extract key attribute information for rule matching, including cloud region identifier, zone identifier, account identifier, device type, data source type, data source version, cloud disk type, cloud disk attributes, cluster identifier, storage pool name, site identifier, and purpose identifier. Simultaneously, the system maintains a directed acyclic graph reflecting the master-slave or cluster relationships between different types of heterogeneous resource instances based on the relationship information contained in the resource instance data. In a directed acyclic graph (DAG), nodes represent specific resource instances, and directed edges represent parent-child or master-slave relationships between these instances, pointing from the parent node to the child node. When a relationship change event between resource instances is synchronized, the system performs atomic operations on the DAG: adding a relationship adds the corresponding directed edge, and removing a relationship removes the corresponding directed edge. Each time the topology of the DAG changes, a globally monotonically increasing topology version number is updated to identify the current topology version.

[0024] In this embodiment, the process of matching automatic grouping rules based on the attribute information of heterogeneous resource instances to be grouped adopts a two-stage matching mechanism. The first stage is the candidate rule screening stage. The system constructs database query conditions based on the resource types of the heterogeneous resource instances to be grouped, and queries the document database for all automatic grouping rules that match the resource types and have at least one non-null condition dimension that intersects with the attribute information of the instances to be grouped. Specifically, for the non-null condition dimensions configured in the rules, query clauses for each dimension are constructed, and all clauses are combined using logical OR relations to form a wide-condition query. This design ensures that potential candidate rules are not missed due to a single dimension mismatch, while avoiding a full table scan of all rules, thus improving the efficiency of rule matching. The second stage is the precise matching verification stage. The system traverses the candidate rule list returned in the first stage and performs inclusion verification on all configured non-null condition dimensions of each rule. For each rule, the following checks are performed sequentially: whether the cloud region identifier of the instance to be grouped is in the rule's cloud region identifier list, whether the zone identifier is in the rule's zone identifier list, whether the account identifier is in the rule's account identifier list, whether the device type is in the rule's device type list, whether the data source type is in the rule's data source type identifier list, whether the data source version is in the rule's data source version identifier list, whether the cloud disk type is in the rule's cloud disk type list, whether the cloud disk attribute is in the rule's cloud disk attribute list, whether the cluster identifier is in the rule's cluster identifier list, whether the storage pool name is in the rule's storage pool name list, whether the site identifier is in the rule's site identifier list, and whether the purpose identifier is in the rule's purpose identifier list. If all configured non-empty condition dimensions of the rule contain the corresponding attribute values ​​of the instance to be grouped, then the instance to be grouped is considered to have matched this rule.

[0025] In this application's embodiments, the process of generating a grouping operation containing a topology snapshot is a key difference between this application and existing technologies. In existing technologies, when an instance matches a grouping rule, the instance group update interface is typically called directly to add the instance to the instance group. This approach has problems in scenarios where resource topology relationships change frequently, because the topology information upon which the grouping decision depends may have changed before being written to the instance group, leading to instances that have lost their original relationships being incorrectly grouped. To solve this problem, this application, after an instance matches a grouping rule, does not directly perform a write operation, but instead generates a grouping operation containing a topology snapshot. A topology snapshot is a static copy of the portion of the topology structure related to the instance to be grouped in the directed acyclic graph at the time the grouping operation is generated. By carrying a topology snapshot in the grouping operation, the topology information upon which the grouping decision depends can be frozen at the time the operation is generated, avoiding the impact of subsequent topology changes on the grouping decision. After the grouping operation is generated, it is encapsulated as a conflict-free copy data type operation. A conflict-free copy data type is a distributed data type that satisfies the commutative, associative, and idempotent laws. The commutative law states that the order in which operations are executed does not affect the final result; the associative law states that the grouping of operations does not affect the final result; and the idempotency law states that executing the same operation multiple times yields the same result as executing it once. These mathematical properties make conflict-free copy data type operations very suitable for asynchronous propagation in distributed systems, ensuring that all nodes eventually obtain a consistent result even if operations are delayed, out of order, or duplicated in the network.

[0026] In this embodiment, the process of propagating grouping and topology change operations as conflict-free replicated data type operations among multiple management nodes can be implemented through message queues or point-to-point communication mechanisms. Each management node in the system maintains a local operation log to record all received conflict-free replicated data type operations. When a management node generates a new conflict-free replicated data type operation, it broadcasts the operation to all other management nodes in the system. Other management nodes, upon receiving the operation, append it to their local operation log. Simultaneously, changes to the directed acyclic graph (DAG) topology are also propagated as conflict-free replicated data type operations. When a management node detects a change in the DAG topology, it generates a corresponding topology change operation and broadcasts it to all other management nodes. The topology change operation also carries the current topology version number to identify the topology version in which the change occurred. By unifying grouping and topology change operations as conflict-free replicated data type operations, the system can handle the propagation and merging of these two types of operations within the same framework, ensuring the consistency of system logic.

[0027] In this embodiment, the process of merging the received conflict-free replication data type operations at each management node achieves eventual consistency. Each management node continuously merges conflict-free replication data type operations in its local operation log. Since conflict-free replication data type operations satisfy the commutative, associative, and idempotent laws, the merging operation can be executed independently at any time and on any node without coordination with other nodes. The merging process merges operations in the operation log according to the operation type and the version number carried, eliminating duplicate and conflicting operations to obtain a consistent set of operations. The merged set of operations reflects the sum of all valid operations occurring on all management nodes. The system periodically materializes the merged set of operations into actual instance group membership relationships and updates the locally maintained instance group state. Since all management nodes have the same merging rules, and the operation set will eventually be synchronized across all nodes, the instance group state on all management nodes will eventually converge to a consistent result.

[0028] In this embodiment, the process of dynamically updating the membership of instance groups based on the merging results is a process of transforming an abstract set of operations into a concrete instance group state. The system iterates through the merged set of operations and updates the instance group's member list according to the type and content of each operation. For a group addition operation, the system adds the corresponding heterogeneous resource instance to the instance group's member list. For a group deletion operation, the system removes the corresponding heterogeneous resource instance from the instance group's member list. For an instance group deletion operation, the system clears the instance group's member list and deletes the instance group itself. During the process of updating instance group membership, the system automatically handles duplicate additions and deletions to ensure the uniqueness and consistency of the instance group's member list. Simultaneously, the system updates relevant monitoring policies and alarm rules based on the latest state of the instance group to ensure that monitoring and alarms accurately cover all resource instances in the instance group.

[0029] In alternative implementations, such as Figure 2 As shown, the grouping operation that generates a topology snapshot includes: 210: Obtain the topology snapshot corresponding to the heterogeneous resource instance to be grouped.

[0030] S220: Generate the grouping and adding operation for the heterogeneous resource instance to be grouped.

[0031] S230: Grouping and adding operations for associated sub-instances of heterogeneous resource instances to be grouped based on topology snapshots.

[0032] In this embodiment, obtaining a topology snapshot corresponding to the heterogeneous resource instance to be grouped is the first step in generating a grouping operation containing the topology snapshot. When the heterogeneous resource instance to be grouped matches the automatic grouping rule, the system requests the topology maintenance module to obtain the topology snapshot corresponding to the instance. The topology maintenance module extracts all nodes and edges that have direct or indirect relationships with the instance to be grouped based on the current directed acyclic graph, generating a static copy of the topology structure. The topology snapshot contains the globally monotonically increasing version number of the current directed acyclic graph, as well as a list of all associated nodes related to the instance to be grouped. The list of associated nodes includes all child nodes of the instance to be grouped, i.e., other resource instances belonging to this instance, such as slave instances of the database master instance, nodes in the cluster, etc. By obtaining the topology snapshot, the system can freeze the topology information on which the grouping decision depends at the moment the operation is generated, ensuring that the grouping operation is generated based on a determined topology state.

[0033] In this embodiment, the process of generating a grouping and adding operation for the heterogeneous resource instance to be grouped is a process of transforming the intention of adding the instance to the instance group into a conflict-free replication data type operation. The system generates a grouping and adding operation based on the identifier of the instance to be grouped, the identifier of the target instance group, the resource type, and the version number of the topology snapshot. The grouping and adding operation belongs to the add-priority set type of conflict-free replication data type operation, whose semantics is to add the specified instance to the specified instance group. A characteristic of the add-priority set type of conflict-free replication data type is that when both add and delete operations exist for the same instance, the add operation takes precedence over the delete operation. This semantics is suitable for grouping operation scenarios because, typically, grouping operations are initiated proactively, while delete operations may be caused by passive reasons such as topology changes; proactive operations should have higher priority.

[0034] In this embodiment, the process of generating grouping and adding operations for associated sub-instances of heterogeneous resource instances to be grouped based on topology snapshots is crucial for achieving master-slave and cluster cascading grouping. For resource types with master-slave or cluster topologies, when a parent instance matches the grouping rule, all its sub-instances should also be automatically grouped into the same instance group to ensure that monitoring and alarm policies can cover the complete topology structure. The system traverses the list of associated nodes in the topology snapshot and generates corresponding grouping and adding operations for each associated sub-instance. Each sub-instance's grouping and adding operation carries the same topology snapshot version number as the parent instance to identify that these operations are generated based on the same topology state. In this way, the grouping operations of parent and sub-instances are associated together, ensuring the consistency of cascading grouping.

[0035] In alternative implementations, such as Figure 3 As shown, obtain the topology snapshot corresponding to the heterogeneous resource instances to be grouped, including: S211: Query the global monotonically increasing version number of the current topological directed acyclic graph.

[0036] S212: Extract a list of associated nodes that have a master-slave or cluster relationship with the heterogeneous resource instance to be grouped.

[0037] S213: Generate a topology snapshot containing the version number and a list of associated nodes.

[0038] In this embodiment, querying the globally monotonically increasing version number of the current topological directed acyclic graph is fundamental to obtaining a topological snapshot. The globally monotonically increasing version number is an integer variable that increments by one each time the topological structure of the directed acyclic graph changes. The monotonically increasing nature of the version number ensures that each topological state has a unique and comparable identifier. The system can determine the order of two topological states by comparing their version numbers. When obtaining a topological snapshot, the system first queries the current topological version number and records it in the topological snapshot as the identifier of the corresponding topological state.

[0039] In this embodiment, the process of extracting the list of associated nodes that have a master-slave or cluster relationship with the heterogeneous resource instance to be grouped involves constructing a topology snapshot. The system finds the node corresponding to the instance to be grouped in the directed acyclic graph, and then performs a depth-first or breadth-first traversal along the directed edges to collect all nodes reachable from that node. These nodes are the associated child instances that have a master-slave or cluster relationship with the instance to be grouped. The traversal process records the identifier, type, and relationship with the parent node of each node. In this way, the system can completely obtain information about all subordinate child instances of the instance to be grouped.

[0040] In this embodiment, generating a topology snapshot containing a version number and a list of associated nodes involves encapsulating the extracted topology information into a static object. The topology snapshot object contains two main fields: a version number field, which stores the globally monotonically increasing version number of the current directed acyclic graph; and an associated node list field, which stores information about all associated child instances that have a master-slave or cluster relationship with the instance to be grouped. After the topology snapshot is generated, it is attached to the grouping operation and propagated to all management nodes along with it. By carrying the topology snapshot in the grouping operation, the system can determine whether the grouping operation is based on outdated topology information based on the version number of the topology snapshot and the version number of the topology change operation during the subsequent merging process, thereby deciding whether to execute the grouping operation.

[0041] In an optional implementation, the received conflict-free replication data type operations are merged at each management node, such as... Figure 4 As shown, it includes: S310: Receives conflict-free replication data type operations from other management nodes and records them in the local operation log.

[0042] S320: Merge operations in the local operation log based on the operation type and the version number carried.

[0043] S330: Determine the grouping status of each heterogeneous resource instance based on the merged set of operations.

[0044] In this embodiment, the process of receiving conflict-free replication data type operations from other management nodes and recording them in the local operation log is fundamental to operation propagation. Each management node maintains a local operation log, which is an append-only log file used to record all received conflict-free replication data type operations. When a management node receives an operation from another node, it first verifies the legality of the operation, including checking whether the operation format is correct and whether the operation signature is valid. After successful verification, the system appends the operation to the end of the local operation log. The append-only nature of the operation log ensures that the order of operations is not changed, and also facilitates system fault recovery and data backup.

[0045] In this embodiment, the process of merging operations in the local operation log based on the operation type and the version number carried is crucial for achieving eventual consistency. The system periodically merges operations in the local operation log, performing merging and conflict resolution based on the operation type and version number. For multiple operations of the same type within the same instance, the system retains the latest operation and discards the older one. For operations of different types within the same instance, the system resolves conflicts according to predefined merging rules. The merging process is idempotent, meaning that multiple executions of the merge operation yield the same result as a single execution. This ensures that even if a system failure occurs during the merging process, re-executing the merge operation will not lead to data inconsistency.

[0046] In this embodiment, the process of determining the grouping status of each heterogeneous resource instance based on the merged operation set is a process of converting the merged operations into instance group states. The system iterates through the merged operation set and maintains a grouping status record for each heterogeneous resource instance. The grouping status record includes information such as instance identifier, instance group identifier, operation type, and operation version number. For each instance, the system determines which instance groups the instance should ultimately belong to based on all operations targeting that instance. If an instance has multiple operations targeting the same instance group, the system determines the final grouping status based on the operation type and version number. In this way, the system can deduce the latest membership relationship of all instance groups from the merged operation set.

[0047] In an optional implementation, operations in the local operation log are merged based on the operation type and the version number carried, such as... Figure 5 As shown, it includes: S321: When both grouping and topology deletion operations exist on the same heterogeneous resource instance, compare the snapshot version number carried by the grouping and addition operation with the topology version number carried by the topology deletion operation.

[0048] S322: If the topology version number of the topology deletion operation is greater than the snapshot version number of the grouping addition operation, then the result of the topology deletion operation is retained.

[0049] S323: If the topology version number of the topology deletion operation is less than or equal to the snapshot version number of the grouping and adding operation, then the result of the grouping and adding operation is retained.

[0050] In this embodiment, when both grouping and topology deletion operations exist for the same heterogeneous resource instance, comparing the snapshot version number carried by the grouping and deletion operations and the topology version number carried by the topology deletion operations resolves conflicts between grouping operations and topology changes. The snapshot version number carried by the grouping and deletion operations is the version number of the directed acyclic graph (DAG) at the time the grouping operation was generated, while the topology version number carried by the topology deletion operations is the version number of the DAG at the time the topology change occurred. By comparing the size of these two version numbers, the system can determine whether the grouping operation is based on outdated topology information. If the topology version number of the topology deletion operation is greater than the snapshot version number of the grouping and deletion operations, it indicates that the topology structure has changed after the grouping operation was generated, and the instance has lost its original hierarchical relationship. Therefore, the grouping operation generated based on the old topology information should be discarded. If the topology version number of the topology deletion operation is less than or equal to the snapshot version number of the grouping and deletion operations, it indicates that the topology change occurred before the grouping operation was generated, and the grouping operation was generated based on the latest topology information. Therefore, the grouping operation should take effect.

[0051] In this embodiment, if the topology version number of the topology deletion operation is greater than the snapshot version number of the grouping addition operation, then retaining the result of the topology deletion operation is a specific step in resolving conflicts. When the system determines that the grouping operation is based on outdated topology information, it will ignore the grouping addition operation and retain the result of the topology deletion operation. This means that the instance will not be added to the corresponding instance group. In this way, the system can automatically filter out erroneous grouping operations based on outdated topology information without executing additional rollback logic. This is significantly different from the distributed transaction and compensation operation methods used in the prior art. In the prior art, when an error in the grouping operation is detected, a compensation operation needs to be performed to remove the instance from the instance group. However, the compensation operation may fail due to network failures, service unavailability, etc., resulting in the system being in an inconsistent state for a long time. This application, through version comparison, directly filters out erroneous grouping operations during the merging phase, avoiding the occurrence of erroneous grouping and also avoiding the complexity and risks brought by the compensation operation.

[0052] In this embodiment, if the topology version number of the topology deletion operation is less than or equal to the snapshot version number of the grouping addition operation, then retaining the result of the grouping addition operation is a step to ensure that the correct grouping operation takes effect. When the system determines that the grouping operation is based on the latest topology information, it will execute the grouping addition operation and add the instance to the corresponding instance group. This ensures that grouping operations that conform to the current topology state can take effect normally. Through this version number-based conflict resolution rule, the system can automatically handle conflicts between grouping operations and topology changes without relying on distributed transactions and a central coordinator, ensuring that the grouping result is always consistent with the latest topology state.

[0053] In optional implementations, attribute information and topology information of different types of heterogeneous resource instances are obtained from multiple cloud platforms, such as... Figure 6 As shown, it includes: S110: Obtain full or incremental heterogeneous resource instance data from resource management services of multiple cloud platforms.

[0054] S120: Generate instance identifiers using corresponding identification strategies for different types of heterogeneous resource instances.

[0055] S130: Maintain a topological directed acyclic graph that reflects the master-slave or cluster relationships between different types of heterogeneous resource instances.

[0056] In this embodiment, the process of obtaining full or incremental heterogeneous resource instance data from resource management services of multiple cloud platforms is the foundation for the system to acquire resource information. The system supports interfacing with multiple different cloud platforms, each providing a corresponding application programming interface (API) for querying resource instance data. The system configures a corresponding adapter for each cloud platform, which is responsible for converting the heterogeneous data format returned by the cloud platform into a unified data format within the system. Full synchronization refers to the system acquiring complete data of all resource instances on the cloud platform at once, typically performed during system initialization or periodic data verification. Incremental synchronization refers to the system acquiring only resource instance data that has changed since the last synchronization, typically performed during daily operation. By combining full and incremental synchronization, the system can improve data synchronization efficiency while ensuring data integrity.

[0057] In this embodiment, the process of generating instance identifiers using corresponding identification strategies for different types of heterogeneous resource instances is crucial for achieving unified management of heterogeneous resources. Different types of cloud resource instances have different identification methods. Some resource instances use globally unique identifiers, some use internal identifiers, and some require a combination of multiple fields for unique identification. To achieve unified management of different types of heterogeneous resource instances, the system adopts differentiated identification strategies. For regular resource instances, the globally unique identifier returned by the cloud platform is directly used as the instance identifier. For database instances, a composite identifier format is used, combining the database's internal identifier and globally unique identifier as the instance identifier. Indexes are established in the cache and relational storage for both the internal identifier and the globally unique identifier to ensure that the instance can be located using either identifier. For cluster-type resource instances, the use of the overall cluster identifier as the instance identifier is supported. Simultaneously, each node within the cluster can be managed as an independent sub-instance, with the sub-instance's identifier composed of the overall cluster identifier and the node identifier. Through this differentiated identification strategy, the system can uniformly handle the identification of different types of heterogeneous resource instances, providing a foundation for subsequent rule matching and grouping operations.

[0058] In this embodiment, maintaining a topological directed acyclic graph (DAG) reflecting the master-slave or cluster relationships between different types of heterogeneous resource instances is fundamental to achieving cascading grouping and conflict resolution. A topological DAG is a data structure used to represent hierarchical relationships between resource instances. The system extracts the master-slave and cluster relationships between resource instances based on resource instance data obtained from the cloud platform and constructs the topological DAG. When the relationships between resource instances change, the system updates the structure of the topological DAG promptly and increments the global topology version number. The maintenance of the topological DAG is atomic; each relationship change is executed as a whole operation, without partial updates. This ensures that the topological DAG remains consistent at all times, providing accurate topological information for grouping operations.

[0059] In an optional implementation, the membership relationships of the instance group are dynamically updated based on the merging results, such as... Figure 7 As shown, it includes: S340: Check if the heterogeneous resource instance to be added already exists in the instance group.

[0060] S350: If the heterogeneous resource instance to be added does not exist in the instance group, then add the instance to the instance group.

[0061] S360: If the heterogeneous resource instance to be added already exists in the instance group, refresh the cache status of that instance.

[0062] In this embodiment, checking whether the heterogeneous resource instance to be added already exists in the instance group is a step to avoid duplicate grouping. Before adding an instance to the instance group, the system queries the current member list of the instance group to check if the instance to be added is already in the member list. If the instance to be added already exists in the member list, the system skips the addition operation to avoid duplicate addition. If the instance to be added does not exist in the member list, the system performs the addition operation and adds the instance to the member list. Through this deduplication mechanism, the system can ensure the uniqueness of the instance group member list and prevent the same instance from appearing multiple times in the same instance group.

[0063] In this embodiment, if the heterogeneous resource instance to be added does not exist in the instance group, the process of adding the instance to the instance group is a step of updating the instance group membership relationship. The system constructs an instance group relationship object, sets fields such as instance identifier, instance group identifier, and resource type, and stores this object in the instance group relationship collection. Simultaneously, the system updates the member count statistics of the instance group, as well as related cached data. After an instance is added to the instance group, it automatically inherits all monitoring policies and alarm rules bound to the instance group. The system configures corresponding monitoring items and alarm thresholds for the instance to ensure that the instance is included in the monitoring scope.

[0064] In this embodiment, if the heterogeneous resource instance to be added already exists in the instance group, refreshing the cached state of that instance is a step to ensure that the cached data is consistent with the actual state. Even if the instance already exists in the instance group, the system will still call the information query interface to obtain the latest state information of the instance and update the data in the cache. This is because the state of the resource instance may change between two grouping operations, such as changes in the instance's attribute information or changes in the instance's running state. By refreshing the cached state, the system can ensure that the data in the cache is always up-to-date, avoiding errors caused by expired cached data.

[0065] In an optional implementation, after dynamically updating the membership relationships of the instance group based on the merging results, such as... Figure 8 As shown, it also includes: S400: Write the association between heterogeneous resource instances and instance groups to the document database using the operation of updating if they exist and inserting if they do not.

[0066] S410: Write the association index between instances and instance groups to the cache database based on the type differences of heterogeneous resource instances.

[0067] S420: Records log information about the automatic grouping operation of heterogeneous resource instances.

[0068] In this embodiment, the process of writing the association between heterogeneous resource instances and instance groups into the document database using an "update if existing, insert if non-existent" operation is a step in achieving relationship persistence. The document database supports atomic operations (update if existing, insert if non-existent), which ensures data consistency even with concurrent writes. When the system needs to write the association between an instance and an instance group into the database, it executes the "update if existing, insert if non-existent" operation. If the association between the instance and the instance group already exists in the database, the relevant fields of the association are updated, such as update time and operation source. If the association does not exist in the database, a new relationship record is inserted. In this way, the system can ensure the persistence of associations while avoiding data inconsistency caused by repeated writes.

[0069] In this embodiment, writing the association index between instances and instance groups to the cache database based on the type differences of heterogeneous resource instances is a step to improve query efficiency. The cache database uses a key-value pair storage structure, which has high read and write performance. The system writes the association index between instances and instance groups to the cache database using different key formats according to the type differences of resource instances. For ordinary resource instances, a fixed prefix plus instance identifier is used as the key, and a set structure is used to store the set of all instance group identifiers to which the instance belongs. For resource instances that require type differentiation, a fixed prefix plus instance identifier plus separator plus resource type is used as the key. For database-type composite identifier resource instances, cache association indexes are established for the internal identifier and the globally unique identifier, respectively. By writing the association index to the cache database, the system can quickly query all instance groups to which a certain instance belongs, or all instances contained in a certain instance group, thus improving the system's response speed.

[0070] In this embodiment, recording the log information of the automatic grouping operation of heterogeneous resource instances is fundamental for operation auditing and problem troubleshooting. The system generates an operation log for each automatic grouping operation, including operation time, operation type, instance identifier, instance group identifier, resource type, operation source, and operation result. The operation logs are persistently stored in a log database for a certain period. Operations personnel can query the operation logs to understand the execution status of the automatic grouping operation, troubleshoot problems that occur during the grouping process, and simultaneously meet audit compliance requirements.

[0071] In alternative implementations, such as Figure 9 As shown, it also includes: S500: When an instance group is deleted, an instance group delete operation is generated and propagated as a conflict-free replication data type operation among multiple management nodes.

[0072] S600: After each management node merges the instance group deletion operation, remove the associated index of the corresponding heterogeneous resource instance from the cache database.

[0073] S700: Delete the documents and heterogeneous resource instance relationship documents corresponding to the instance group in the document database. Update the alarm policy bound to the instance group.

[0074] In this embodiment, when an instance group is deleted, the process of generating an instance group deletion operation and propagating it as a conflict-free replicated data type operation among multiple management nodes is the first step in achieving cascading cleanup of instance groups. When an operations and maintenance personnel delete an instance group through the user interface, the system does not directly execute the deletion operation, but instead generates an instance group deletion operation. The instance group deletion operation is a conflict-free replicated data type operation, carrying the instance group identifier and the operation generation time. This operation is broadcast to all management nodes in the system, and all management nodes, upon receiving the operation, will execute the corresponding cascading cleanup operation. In this way, the system can ensure that instance group data on all management nodes is consistently deleted.

[0075] In this embodiment, after each management node merges the instance group deletion operation, removing the associated indexes of the corresponding heterogeneous resource instances in the cache database is a crucial step in the cascading cleanup. The system queries all instance identifiers contained in the instance group, then iterates through these instance identifiers, removing the associated indexes of each instance and the instance group from the cache database. For database-type composite identifier resource instances, the associated indexes corresponding to the internal identifier and the globally unique identifier need to be removed separately. By removing the cache indexes, the system can ensure that subsequent query operations will no longer return information about the deleted instance group.

[0076] In this embodiment, deleting the documents corresponding to the instance group and the heterogeneous resource instance relationship documents in the document database is a cascading cleanup step. The system deletes the documents corresponding to the instance group, as well as the association relationship documents between the instance group and all instances, from the document database. Simultaneously, the system deletes the automatic grouping rule documents configured for the instance group to prevent subsequent instances from being automatically grouped into a deleted instance group. By deleting these documents, the system can thoroughly clean up all persistent data related to the instance group.

[0077] In this embodiment, updating the alarm policy bound to the instance group is a step to ensure the normal operation of the alarm system. The system queries all alarm policies bound to the instance group and removes the instance group's identifier from the instance group identifier list of each alarm policy. If the instance group identifier list of an alarm policy becomes empty, the system will prompt the operations and maintenance personnel that the alarm policy no longer has an associated instance group and needs to be adjusted accordingly. At the same time, the system will batch set all alarms occurring under the corresponding alarm policy for all instances in the instance group to an invalid state and generate an invalidation reason flag. By updating the alarm policy, the system can avoid false alarms caused by the deletion of the instance group.

[0078] Another aspect of this application proposes a dynamic automatic grouping system for heterogeneous resource instances in a multi-cloud environment, such as... Figure 10 As shown, the system includes: The rule management module 11 is used to configure automatic grouping rules for instance groups and obtain attribute information and topology information of heterogeneous resource instances of different types from multiple cloud platforms.

[0079] The synchronous propagation module 12 is used to match automatic grouping rules based on the attribute information of the heterogeneous resource instances to be grouped, generate a grouping operation containing a topology snapshot, and propagate the grouping operation and topology change operation as conflict-free replication data type operations among multiple management nodes. Conflict-free replication data type refers to a distributed data type that satisfies the commutative law, associative law and idempotency law.

[0080] The merge update module 13 is used to merge the received conflict-free replication data type operations at each management node and dynamically update the membership relationship of the instance group based on the merge result.

[0081] Based on the same principle, the dynamic automatic grouping system for heterogeneous resource instances in a multi-cloud environment described in this application can be found in the implementation of the above method, and will not be elaborated upon here.

[0082] For example, the implementation process of the technical solution of this application will be described in detail below, taking a scenario of cascading relational database instances in a multi-cloud environment. The system is deployed on three management nodes, namely Node 1, Node 2, and Node 3. Each management node runs a complete rule management module, data synchronization module, operation generation module, propagation module, merging module, and update module. The three management nodes share the same set of document database and cache database, or each maintains local storage and achieves eventual consistency through conflict-free replication of data types. This embodiment adopts a method in which each node independently writes to shared storage.

[0083] In this embodiment, rule configuration and data synchronization are performed first. Operations personnel create an instance group named "Transaction Database" through the resource management platform's user interface and configure an automatic grouping rule. The resource type of this rule is set to relational database, and the purpose identifier is set to "transaction." After the rule configuration is complete, it is stored in the document database. The data synchronization modules of the three management nodes obtain data from the relational database instances from the resource management services of multiple cloud platforms through scheduled synchronization. Simultaneously, the data synchronization modules maintain a topological directed acyclic graph reflecting the master-slave relationships between the relational database instances. Initially, the version number of the topological directed acyclic graph is 100, containing one master instance and two read-only slave instances. A master-slave relationship exists between the master instance and the two slave instances, and the topological directed acyclic graph has two directed edges, one from the master instance and one from the slave instance.

[0084] In this embodiment, after the data synchronization module of node two obtains the latest data of the main instance, it sends the attribute information of the main instance to the operation generation module. The two-stage matching engine of the operation generation module performs rule matching on the attribute information of the main instance. In the first stage, the matching engine constructs database query conditions based on the resource type of the main instance (relational database) and queries all automatic grouping rules of the resource type (relational database) from the document database. The query result returns the rule corresponding to the transaction database instance group configured earlier. In the second stage, the matching engine performs precise matching verification on the rule. The purpose identifier dimension of the rule is set to transaction, and the purpose identifier of the main instance is also transaction. Other dimensions have no attribute values ​​set, so all configured dimensions match successfully. The matching engine determines that the main instance matches the rule.

[0085] In this embodiment, after the operation generation module determines that the master instance matches the rule, it requests the data synchronization module to obtain the topology snapshot corresponding to the master instance. The data synchronization module queries the global version number of the current directed acyclic graph, which is 100, and then extracts the list of associated nodes that have a master-slave relationship with the master instance, i.e., the two slave instances. The data synchronization module generates a topology snapshot containing the version number 100 and the list of associated nodes, and returns it to the operation generation module. Based on the topology snapshot, the operation generation module generates three grouping and adding operations. The first operation is the grouping and adding operation of the master instance itself, carrying the instance group identifier, master instance identifier, resource type, and snapshot version number 100. The second operation is the grouping and adding operation of the first slave instance, carrying the same instance group identifier, the identifier of the first slave instance, resource type, and snapshot version number 100. The third operation is the grouping and adding operation of the second slave instance, carrying the same instance group identifier, the identifier of the second slave instance, resource type, and snapshot version number 100. The operation generation module encapsulates these three grouping and adding operations into conflict-free replication data type operations and sends them to the propagation module. The propagation module broadcasts these three operations to node one, node two, and node three.

[0086] In this embodiment, a master-slave switch event occurs on the cloud platform momentarily after snapshot generation and before operation broadcasting is completed. The second slave instance is promoted to the new master instance, and the master-slave relationship between the original master instance and the second slave instance is terminated. The data synchronization module of node one is the first to receive this master-slave switch event via event listening. The data synchronization module updates the topology directed acyclic graph, removes the directed edge from the original master instance to the second slave instance, and increments the global topology version number to 101. Simultaneously, the data synchronization module generates a topology deletion operation, which is a relationship deletion operation, carrying the original master instance identifier, the second slave instance identifier, and the topology version number 101. The data synchronization module encapsulates this topology deletion operation into a conflict-free replication data type operation and sends it to the propagation module. The propagation module broadcasts this topology deletion operation to nodes one, two, and three.

[0087] In this embodiment, the merging modules of the three management nodes successively receive three grouping add operations and one topology delete operation. Each merging module records the received operations in its local operation log and performs the merging operation periodically. For the grouping add operation of the primary instance and the first slave instance, there is no corresponding topology delete operation, so the merging result retains these two add operations. For the grouping add operation and the topology delete operation of the second slave instance, the merging module compares the version numbers carried by the two operations. The snapshot version number of the grouping add operation is 100, and the topology version number of the topology delete operation is 101. Since 101 is greater than 100, according to the merging rules, the topology delete operation overrides the grouping add operation, and the merging result retains the result of the topology delete operation, that is, the second slave instance will not be added to the transaction database instance group.

[0088] In this embodiment, the update modules of the three management nodes dynamically update the membership of the transaction database instance group based on the merged operation set. The update module checks whether the primary instance and the first secondary instance already exist in the instance group. Initially, the instance group is empty, so neither instance exists. The update module adds the primary instance and the first secondary instance to the instance group's member list. Simultaneously, the update module writes the association between the two instances and the instance group to the document database using an "update if existing, insert if non-existent" operation, and writes the association index to the cache database based on resource type differences. The update module also records log information for the automatic grouping operation, including operation time, operation type, instance identifier, and instance group identifier. Because the merging results of the three management nodes are consistent, the membership of the transaction database instance group on all three management nodes ultimately includes the primary instance and the first secondary instance, but not the second secondary instance. The entire process involves no distributed transaction coordination, no rollback operations, and all nodes eventually converge to a consistent state.

[0089] In this embodiment, after a period of time, the operations and maintenance personnel decide to delete the transaction database instance group. The personnel execute the deletion operation through the user interface, and the system generates an instance group deletion operation, carrying the instance group identifier and the operation generation time. This operation is broadcast to three management nodes. Upon receiving the instance group deletion operation, the merging module of each management node merges it into its local operation set. Based on the merging result, the update module performs a cascading cleanup operation. The update module first queries all instance identifiers contained in the transaction database instance group, namely the primary instance and the first secondary instance. Then, the update module removes the association indexes between these two instances and the transaction database instance group from the cache database. Next, the update module deletes the documents of the transaction database instance group, the documents relating instances to the instance group, and the automatic grouping rule documents associated with the instance group from the document database. The update module also queries all alarm policies bound to the transaction database instance group, removes the instance group identifier from the instance group identifier list of each alarm policy, and batch-sets all active alarms under the corresponding alarm policy for all instances in the instance group to an invalid state. Through these steps, the system completes the cascading cleanup of the instance group, and all related data on all management nodes is consistently deleted.

[0090] It's important to note that existing technologies typically employ distributed transactions or locking mechanisms to address consistency issues in distributed systems. Distributed transactions use two-phase commit or three-phase commit protocols to ensure that operations on multiple nodes either all succeed or all fail. However, distributed transactions suffer from poor performance, low scalability, and susceptibility to blocking, and can lead to data inconsistency in the event of network partitions. Locking mechanisms acquire locks before operating on shared resources to ensure that only one node can operate on shared resources at a time. However, locking mechanisms reduce concurrency and are prone to deadlocks. To address the problems of distributed transactions and locking mechanisms, some systems employ eventual consistency models, using asynchronous replication and compensation operations to guarantee eventual data consistency. However, the compensation operation itself may fail, leading to a prolonged inconsistent state. Furthermore, in scenarios with frequent changes in resource topology, the logic of the compensation operation becomes extremely complex and difficult to maintain.

[0091] This application departs from conventional approaches in existing technologies, instead unifying the grouping intent and topology change events into a unified, conflict-free replication data type operation. The conflict-free replication data type is specifically designed for distributed systems; through predefined merging rules, it guarantees that all nodes will eventually achieve a consistent result without a central coordinator. The mathematical properties of the conflict-free replication data type make it well-suited for use in asynchronous, unreliable network environments. Furthermore, this application models the topology as a directed acyclic graph with version numbers. By carrying a topology snapshot and version number in the grouping operation, it achieves a semantic association between the grouping intent and topology changes. When the topology changes, the grouping operation generated based on the old snapshot is automatically overridden by the topology change operation carrying a higher version number, eliminating the need for additional rollback logic. This combination of technical features is not a simple technological overlay but fundamentally changes the consistency guarantee mechanism of automatic grouping systems. In existing technologies, grouping operations and topology changes are typically treated as two independent processes. The grouping operation only focuses on instance attribute information, not topology information, and topology changes do not affect already executed grouping operations. This leads to the problem of incorrect grouping in scenarios with frequent topology changes. This application unifies grouping operations and topology changes under the same conflict-free replicated data type framework, automatically handling conflicts between the two through version comparison, ensuring that the grouping result is always consistent with the latest topology state. This design approach is unprecedented in existing technologies; it avoids the complexity and risks associated with distributed transactions while also improving the system's concurrency and scalability.

[0092] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0093] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for dynamic automatic grouping of heterogeneous resource instances in a multi-cloud environment, characterized in that, The method includes: Configure automatic grouping rules for instance groups to obtain attribute information and topology information of different types of heterogeneous resource instances from multiple cloud platforms; Based on the attribute information of the heterogeneous resource instances to be grouped, the automatic grouping rules are matched to generate a grouping operation that includes a topology snapshot. The grouping operation and the topology change operation are propagated among multiple management nodes as conflict-free replication data type operations. Conflict-free replication data type refers to a distributed data type that satisfies the commutative law, associative law and idempotency law. At each management node, merge the received conflict-free replication data type operations and dynamically update the membership relationship of the instance group based on the merge result; Generate a grouping operation that includes a topology snapshot, including: Query the globally monotonically increasing version number of the current directed acyclic graph; Extract a list of associated nodes that have a master-slave or cluster relationship with the heterogeneous resource instances to be grouped; Generate a topology snapshot that includes the version number and a list of associated nodes; The received conflict-free replication data type operations are merged at each management node, including: Receive conflict-free replication data type operations from other management nodes and record them in the local operation log; Merge operations in the local operation log based on the operation type and the version number carried. The grouping status of each heterogeneous resource instance is determined based on the merged set of operations; Merge operations in the local operation log based on the operation type and the version number carried, including: When both grouping and topology deletion operations exist on the same heterogeneous resource instance, compare the snapshot version number carried by the grouping and topology deletion operation with the topology version number carried by the topology deletion operation. If the topology version number of the topology deletion operation is greater than the snapshot version number of the grouping and adding operation, then the result of the topology deletion operation is retained. If the topology version number of the topology deletion operation is less than or equal to the snapshot version number of the grouping addition operation, then the result of the grouping addition operation is retained.

2. The method according to claim 1, characterized in that, The grouping operation that generates a topology snapshot also includes: Generate the grouping and adding operation for the heterogeneous resource instance to be grouped; The grouping and addition operation is performed based on the topology snapshot to generate associated sub-instances of heterogeneous resource instances to be grouped.

3. The method according to claim 1, characterized in that, Obtain attribute and topology information of different types of heterogeneous resource instances from multiple cloud platforms, including: Obtain full or incremental heterogeneous resource instance data from resource management services of multiple cloud platforms; Instance identifiers are generated using corresponding identification strategies for different types of heterogeneous resource instances. Maintain a topological directed acyclic graph that reflects the master-slave or cluster relationships between different types of heterogeneous resource instances.

4. The method according to claim 1, characterized in that, The membership relationships of the instance group are dynamically updated based on the merge results, including: Check if the heterogeneous resource instance to be added already exists in the instance group; If the heterogeneous resource instance to be added does not exist in the instance group, then add the instance to the instance group; If the heterogeneous resource instance to be added already exists in the instance group, then refresh the cache status of that instance.

5. The method according to claim 1, characterized in that, After dynamically updating the membership of the instance group based on the merge results, it also includes: Write the association between heterogeneous resource instances and instance groups into the document database by updating if they exist and inserting if they do not. Write the association index between instances and instance groups to the cache database based on the type differences of heterogeneous resource instances; Record log information about the automatic grouping operation of heterogeneous resource instances.

6. The method according to claim 1, characterized in that, Also includes: When an instance group is deleted, an instance group deletion operation is generated and propagated as a conflict-free replication data type operation among multiple management nodes. After each management node merges the instance group deletion operation, remove the associated index of the corresponding heterogeneous resource instance from the cache database; Delete the documents and heterogeneous resource instance relationship documents corresponding to the instance group in the document database; update the alarm policy bound to the instance group.

7. A dynamic automatic grouping system for heterogeneous resource instances in a multi-cloud environment, characterized in that, The system includes: The rules management module is used to configure automatic grouping rules for instance groups and obtain attribute information and topology information of different types of heterogeneous resource instances from multiple cloud platforms. The synchronous propagation module is used to match automatic grouping rules based on the attribute information of the heterogeneous resource instances to be grouped, generate grouping operations that include topology snapshots, and propagate the grouping operations and topology change operations as conflict-free replication data type operations among multiple management nodes. Conflict-free replication data type refers to a distributed data type that satisfies the commutative law, associative law, and idempotency law. Generate a grouping operation that includes a topology snapshot, including: Query the globally monotonically increasing version number of the current directed acyclic graph; Extract a list of associated nodes that have a master-slave or cluster relationship with the heterogeneous resource instances to be grouped; Generate a topology snapshot that includes the version number and a list of associated nodes; The merge update module is used to merge the received conflict-free replication data type operations at each management node and dynamically update the membership relationship of the instance group based on the merge result. The received conflict-free replication data type operations are merged at each management node, including: Receive conflict-free replication data type operations from other management nodes and record them in the local operation log; Merge operations in the local operation log based on the operation type and the version number carried. The grouping status of each heterogeneous resource instance is determined based on the merged set of operations; Merge operations in the local operation log based on the operation type and the version number carried, including: When both grouping and topology deletion operations exist on the same heterogeneous resource instance, compare the snapshot version number carried by the grouping and topology deletion operation with the topology version number carried by the topology deletion operation. If the topology version number of the topology deletion operation is greater than the snapshot version number of the grouping and adding operation, then the result of the topology deletion operation is retained. If the topology version number of the topology deletion operation is less than or equal to the snapshot version number of the grouping addition operation, then the result of the grouping addition operation is retained.

Images