A cloud resource scheduling method and system based on a multi-core architecture

Abstract

The application relates to the technical field of distributed computing resource scheduling, and discloses a Xinchuang cloud resource scheduling method and system based on a multi-core architecture. The method comprises the following steps: collecting load data and chip integration information to generate real-time monitoring results; performing node grouping by using a load balancing algorithm according to the monitoring results to determine resource allocation requirements; when the node load exceeds a preset threshold, triggering a resource virtualization technology to perform resource remapping to obtain node resource configuration; constructing a cluster structure based on the new configuration and determining a resource reconstruction scheme; performing data migration according to the reconstruction scheme, generating a migration path by using a compatibility interface algorithm and verifying state synchronization integrity; when the synchronization is complete, activating a dynamic management strategy to optimize a resource allocation model. The application can improve the resource utilization rate and parallel processing capacity of the multi-core cluster, solves the problems of low efficiency and stability caused by uneven resource distribution and load surge in the Xinchuang cloud environment, and realizes efficient and self-controllable utilization of resources.

Description

Technical Field

[0001] This application relates to the field of distributed computing resource scheduling technology, and in particular to a method and system for scheduling domestically developed cloud resources based on a multi-core architecture. Background Technology

[0002] With the rapid development of cloud computing technology, resource scheduling and management have become core issues in ensuring the efficient operation of cloud platforms. Traditional cloud computing resource scheduling methods mainly rely on single or a few core computing nodes, which cannot fully utilize the computing power and processing efficiency of multi-core architectures. Especially in cloud computing systems integrated with domestically produced chips, due to the special nature of the hardware architecture and technical limitations, how to rationally allocate computing resources, balance the load, and ensure the high availability and security of the system has become a major challenge facing current technology.

[0003] With the development of the information technology innovation industry, more and more cloud computing platforms are adopting domestically produced chips based on multi-core architectures. However, existing cloud resource scheduling methods are not fully adapted to this new hardware environment. Traditional scheduling methods mostly rely on static load balancing, lacking dynamic adjustment and real-time optimization mechanisms for multiple data streams and node resources. This leads to uneven distribution of computing resources and may even cause system bottlenecks due to overloaded nodes, affecting the processing capacity and stability of the cloud platform.

[0004] To adapt to the characteristics of multi-core architectures, current research focuses on how to intelligently schedule cloud computing resources through dynamic monitoring and real-time data analysis. While existing technologies have made breakthroughs in some aspects, problems remain, such as insufficient resource utilization in multi-core architectures, inadequate dynamic load balancing, and imperfect security verification. Therefore, developing a resource scheduling method for domestically developed cloud computing in multi-core architecture environments to solve problems such as uneven resource allocation, overload, and data migration integrity, while improving cluster stability and scalability, has become a critical issue that urgently needs to be addressed in the field. Summary of the Invention

[0005] This invention aims to overcome the problems of low resource utilization, poor load balancing, incomplete data migration, and insufficient system stability in existing cloud computing resource scheduling methods under multi-chip architecture environments. It proposes a cloud resource scheduling method and system based on multi-chip architecture to achieve efficient scheduling and dynamic management of cloud resources under domestic chip environments.

[0006] Firstly, this application provides a method for scheduling cloud resources based on a multi-core architecture, the method comprising:

[0007] Step 1: By using the agent module deployed on the computing nodes, collect the load data and chip integration information of each node, generate real-time monitoring results, and determine the resource distribution among the nodes;

[0008] Step 2: Based on the real-time monitoring results, apply the load balancing algorithm to group the computing nodes, obtain the node classification results, and determine the resource allocation requirements of each node.

[0009] Step 3: If the node classification results show that the load value of a certain type of node exceeds the preset threshold, the resource adjustment mechanism is triggered. The node resources are remapped through resource virtualization technology to obtain the adjusted node resource configuration.

[0010] Step 4: Construct a new cluster structure based on the adjusted node resource configuration, extract reconstruction parameters, and determine a resource reconstruction scheme based on the reconstruction parameters;

[0011] Step 5: Execute the data migration process according to the resource restructuring plan, generate the migration path using the compatibility interface algorithm, and determine the integrity of the state synchronization during the migration process based on the migration path;

[0012] Step 6: If the state synchronization integrity is confirmed, the dynamic management strategy is activated and a resource allocation model is generated, thereby achieving stable allocation of system resources and overall operational stability.

[0013] Secondly, this application provides a cloud resource scheduling system based on a multi-core architecture, the system comprising:

[0014] The data acquisition module is used to collect load data and chip integration information of each node through the agent module deployed on the computing node, generate real-time monitoring results, and determine the resource distribution between nodes.

[0015] The grouping evaluation module is used to group computing nodes based on real-time monitoring results using a load balancing algorithm, obtain node classification results, and determine the resource allocation requirements of each node.

[0016] The resource adjustment module is used to trigger a resource adjustment mechanism when the load value of a certain type of node exceeds a preset threshold, based on the node classification results. The module uses resource virtualization technology to remap the node resources and obtain the adjusted node resource configuration.

[0017] The structure building module is used to build a new cluster structure based on the adjusted node resource configuration, extract reconstruction parameters, and determine the resource reconstruction scheme based on the reconstruction parameters.

[0018] The migration execution module is used to execute the data migration process according to the resource reconstruction plan, generate the migration path using a compatibility interface algorithm, and determine the integrity of the state synchronization during the migration process based on the migration path.

[0019] The strategy activation module is used to activate dynamic management strategies and generate resource allocation models when the integrity of state synchronization is confirmed, thereby achieving stable allocation of system resources and overall operational stability.

[0020] Compared with the prior art, the beneficial effects of the technical solution of this application are at least as follows:

[0021] 1. By introducing a dynamic load balancing algorithm based on real-time monitoring data, automatic allocation and rebalancing of resources among nodes can be achieved in a multi-core architecture, avoiding overload or idle conditions of some nodes and significantly improving resource utilization and load balancing.

[0022] 2. By utilizing resource virtualization technology to achieve dynamic remapping of node resources, resource reallocation can be completed without interrupting operation. At the same time, combined with data synchronization based on cache consistency protocol and high-speed transmission of interconnect bus, the consistency of cluster state during adjustment is ensured, thereby building a highly available cluster structure with redundant nodes, improving the system's flexibility and scalability, and adapting to the real-time adjustment needs under different load environments.

[0023] 3. By combining the compatibility interface algorithm with the security encryption module, protocol compatibility, path optimization and data integrity verification are achieved during the data migration process, preventing migration interruption and data loss, and ensuring the security of data migration and the reliability of state synchronization.

[0024] 4. By introducing dynamic management strategies and optimization algorithms, the system can autonomously adjust resource allocation parameters, enabling it to maintain efficient operation under load changes or hardware differences, thereby achieving long-term stable scheduling and high availability in the domestic cloud environment. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the cloud resource scheduling method based on a multi-core architecture according to this application;

[0027] Figure 2 This is a schematic diagram of the domestically developed cloud resource scheduling system based on a multi-core architecture as described in this application. Detailed Implementation

[0028] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] For ease of understanding, the specific process of the embodiments of this application is described below. Figure 1 The diagram shows a flowchart of an embodiment of the cloud resource scheduling method based on a multi-core architecture provided by the present invention. The flowchart specifically includes the following steps:

[0030] Step 1: By using the agent module deployed on the computing nodes, collect the load data and chip integration information of each node, generate real-time monitoring results, and determine the resource distribution among the nodes.

[0031] In one specific embodiment, the process of performing step 1 may specifically include the following steps:

[0032] By using agent modules deployed on each computing node, load data and chip integration information of the computing node are periodically collected through a polling mechanism.

[0033] Based on load data, a weighted average method is used to calculate the resource utilization of each computing node;

[0034] Based on chip integration information, the parallel processing capability data of each computing node is analyzed and extracted;

[0035] By integrating resource utilization and parallel processing capability data, structured real-time monitoring results are generated.

[0036] Based on real-time monitoring results, the variance or standard deviation of resource utilization rates among nodes is calculated to determine whether there is an uneven distribution of resources among the nodes.

[0037] If it is determined that there is an uneven distribution of resources, the scalability configuration requirements are determined based on the degree of unevenness and historical trends. The scalability configuration requirements include the number of new nodes, virtual resource mapping strategies and / or parameters for improving parallel capabilities.

[0038] Based on parallel processing capabilities, a multi-threaded parallel algorithm is used to process multiple monitoring data streams and summarize them to generate an analysis report that includes resource distribution statistics and optimization suggestions.

[0039] Specifically, the agent module periodically extracts load data such as CPU utilization and memory usage from the computing node operating system using a polling mechanism, and reads the integrated information of the domestically produced chip from the chip firmware, including the number of cores, clock frequency, and parallel processing capabilities. In this way, the agent module ensures real-time performance and obtains comprehensive performance data for each node.

[0040] After obtaining the load data, the agent module calculates the resource utilization of each computing node using a weighted average method. Resource utilization refers to the degree to which a computing node uses its available resources within a certain period, specifically including the usage of resources such as CPU and memory. By using a weighted average method, the impact of different types of data (such as CPU utilization and memory usage) on resource utilization is taken into account, thus more accurately reflecting the resource utilization of the nodes. Based on this, the agent module further extracts and analyzes the parallel processing capability data of each computing node according to the chip integration information, including key indicators such as the number of parallel threads supported and SIMD instruction set capabilities.

[0041] Subsequently, the agent module integrates resource utilization and parallel processing capability data to generate structured real-time monitoring results. This data not only includes the resource usage of each node but also reflects its parallel processing capability, ensuring a comprehensive assessment of node resources. Based on these real-time monitoring results, the agent module calculates the variance or standard deviation of resource utilization among nodes to determine whether the resource distribution is balanced. Variance or standard deviation calculations quantify the degree of deviation in node resource distribution, thereby identifying any imbalances. If the variance or standard deviation exceeds a corresponding threshold, it indicates an imbalance. The agent module then further determines scalability requirements based on the degree of imbalance (e.g., the difference between the calculated result and the corresponding threshold) and historical trends, including the number of new nodes, virtual resource mapping strategies, or improvements in parallel processing capabilities. Historical trend data refers to the changing trend of computing node load over a past period, typically obtained by collecting load information from nodes at different times. This data helps the system predict future load changes and provides a basis for decisions regarding scalability requirements.

[0042] After determining the degree of imbalance and historical trends, the system will automatically calculate and generate specific scalability configuration requirements. For example, when resources are unbalanced and load surges, the system will calculate the number of new nodes needed based on the degree of imbalance and historical trends. This requirement depends on the differences in node load. For instance, if existing nodes cannot meet the increased load demand, the system will simulate expansion scenarios to determine how many new nodes to add to share the load. When the number of nodes increases or node resources are insufficient, virtualization technology can be used to map physical resources to a virtual resource pool, providing more flexible resource configuration. The determination of the virtual resource mapping strategy is based on the node... When node computing resources are insufficient, the system flexibly allocates computing, memory, and storage resources through resource mapping strategies. For example, when a node is overloaded, virtualization can migrate tasks to idle nodes, optimizing the utilization of existing resources. In some cases, the parallel processing capabilities of existing nodes may not be sufficient to support new load demands, especially in scenarios requiring the processing of a large number of parallel tasks. Therefore, the system calculates parameters to improve parallel processing capabilities based on existing parallel processing capability data. For instance, if the number of parallel threads on a node does not meet the required computing capacity, the system may improve its processing capacity through hardware upgrades or by increasing the number of parallel computing threads. For example, with an imbalance of 25% and a scalability configuration requirement of adding two nodes, each with at least eight parallel threads, the example initially had 10 nodes. After adding these nodes, the total parallel processing capability increases by 20%, which is beneficial for load scalability.

[0043] Furthermore, to enhance the processing capability of multi-channel monitoring data streams, the agent module employs a multi-threaded parallel algorithm based on parallel processing capability data to process these data streams, thereby efficiently summarizing and generating an analysis report containing resource distribution statistics and optimization suggestions. The multi-threaded parallel algorithm can run the computational tasks of multiple data streams simultaneously on multiple processing cores, ensuring high efficiency in the processing process, especially when handling large-scale data streams, significantly improving processing speed and computational efficiency. Through this analysis report, the system can generate optimization suggestions for each node based on real-time monitoring results, and further adjust resource allocation strategies accordingly to ensure a balanced distribution of system resources.

[0044] This invention's technical solution, through real-time data acquisition by the agent module, calculation of resource utilization using a weighted average method, and analysis and fusion of parallel processing capabilities, combined with variance and standard deviation to calculate resource imbalance, ultimately generates an optimized resource distribution report. This solves problems such as inaccurate resource scheduling, uneven load, and insufficient data stream processing capabilities. This technical approach enables precise scheduling and optimization of cloud computing resources in multi-core architectures, improving system performance and enhancing resource utilization efficiency, thereby supporting a high-efficiency and stable cloud computing environment.

[0045] In a preferred embodiment, based on parallel processing capability data, a multi-threaded parallel algorithm is used to process multiple monitoring data streams, and an analysis report containing resource distribution statistics and optimization suggestions is generated, including:

[0046] For multiple monitoring data streams, the processing priority of each data stream is calculated based on the data stream size and arrival time, and a priority queue is formed.

[0047] Retrieve data streams from the priority queue, query available computing core resources, and distribute the core resources evenly to each data stream;

[0048] Based on the resource allocation results, perform parallel processing of multiple data streams, record the core usage time and memory consumption data of each data stream, and generate a resource distribution data table for each node based on the recorded execution results.

[0049] Based on the resource distribution data table, calculate the standard deviation of core usage time and memory usage among nodes, and obtain a comprehensive balance score based on a weighted average.

[0050] If the overall balance score exceeds the preset balance threshold, resource distribution optimization suggestions are generated. The optimization suggestions include the source node to migrate, the target node to migrate, and the migration data stream identifier.

[0051] Based on the optimization suggestions, the processing strategy for multiple data streams was adjusted, and the priority queue and data stream allocation were updated.

[0052] The report summarizes the adjusted resource distribution data, overall balance score, optimization suggestions, and the results of implementing the optimization suggestions, and generates an analysis report.

[0053] Specifically, for multiple monitoring data streams, the processing priority of each data stream is first calculated by collecting its size and arrival time. The priority value is calculated by dividing the data stream's size by its arrival time, resulting in a value reflecting the importance and urgency of the data stream. Data streams with higher priority values ​​require priority processing. Next, all data streams are sorted according to their priority values ​​to form a priority queue. This sorting ensures that, under heavy load, the most important or urgent data streams are processed first, preventing low-priority data streams from blocking the system.

[0054] After determining the processing priority of each data stream, resource allocation is performed by querying available computing core resources. Based on parallel processing capability data, data streams are retrieved from the priority queue, and core resources are evenly distributed to each data stream according to the current number of computing cores. In this process, the system combines the core resource allocation strategy with the priority of the data streams to ensure that high-priority data streams receive sufficient computing resources, thereby improving the system's processing efficiency.

[0055] Based on the allocation results, parallel processing is initiated, and the allocated computing cores are bound to the corresponding data streams. Data computation is completed through multi-threaded processing. During this process, each data stream runs independently on multiple computing cores, enabling the simultaneous processing of large amounts of data and significantly improving processing efficiency. During parallel processing, the core usage time and memory consumption of each data stream are recorded. By aggregating this data, a resource distribution data table between nodes is generated.

[0056] Further analysis of resource distribution data is used to calculate the resource distribution balance of each node based on the standard deviation of core usage time and memory usage. A large standard deviation indicates an uneven resource allocation. Specifically, the average core usage time and memory usage for each node are first extracted from the distribution data table. The average core usage time is obtained by summing the core usage time of all data streams on that node and dividing by the number of data streams. The memory usage is the average of peak memory usage during processing. Then, the standard deviations of core usage time and memory usage for each node are calculated separately. A weighted average of the two standard deviations is then calculated to obtain a comprehensive balance score. If the comprehensive balance score is less than a set threshold (e.g., 0.5), the resource distribution among nodes is considered balanced; otherwise, it is considered unbalanced.

[0057] If the overall load balance score exceeds a preset load balance threshold, nodes with high load are identified from the resource distribution data table. Nodes with high load are determined by comparing their average core usage time to the overall average and finding it exceeds a certain percentage (e.g., 20%). Then, it is recommended to migrate a portion of the data flow to nodes with lower load, thereby balancing the load across nodes. The migration amount is calculated based on a percentage of the standard deviation; for example, with a standard deviation of 1.29, approximately 10% of the load is recommended to be migrated. The generated list of optimization recommendations includes the source node, the target node, and the identifier of the data flow to be migrated.

[0058] Based on these optimization suggestions, the processing strategy for multiple data streams is adjusted, the priority queue is updated, and the data streams are reallocated. The new resource allocation will be more balanced, ensuring efficient data stream processing and rational resource utilization. Finally, the adjusted resource distribution data, comprehensive balance score, optimization suggestions, and the results of implementing the optimization suggestions are summarized to generate an analysis report on the resource distribution among nodes.

[0059] By employing multi-threaded parallel processing and priority scheduling strategies, the processing efficiency of the data stream is significantly improved, avoiding uneven resource allocation and processing bottlenecks. Furthermore, balance analysis based on standard deviation and weighted average methods can identify and resolve resource imbalances between nodes in real time, thereby enhancing system stability and scalability. By generating optimization suggestions and dynamically adjusting resource allocation strategies, this invention's technical solution can achieve reasonable resource scheduling under high load conditions, ensuring efficient system operation.

[0060] Step 2: Based on the real-time monitoring results, apply the load balancing algorithm to group the computing nodes, obtain the node classification results, and determine the resource allocation requirements of each node.

[0061] In one specific embodiment, the process of performing step 2 may specifically include the following steps:

[0062] The load parameters of each computing node are extracted based on the real-time monitoring results, and the load parameters of each computing node are weighted and summed to obtain the load value of each computing node.

[0063] The load value is compared with a preset load threshold, and the computing nodes are classified into load surge nodes and idle nodes based on the comparison results.

[0064] For nodes experiencing a surge in load, the resource allocation requirement is calculated based on the node's current load value and parallel processing capacity data.

[0065] For idle nodes, the amount of available resources is determined based on their remaining computing resources and available bandwidth;

[0066] The cache consistency protocol maintains the synchronization of packet data between nodes experiencing load surges and idle nodes, including writing packet data to a shared cache, broadcasting update messages, and employing a write failure strategy to ensure data consistency.

[0067] Based on the consistency rate of grouped data synchronization, determine whether the activation conditions of the thread scheduling mechanism are met;

[0068] When the activation conditions are met, the thread migration priority is calculated based on the resource allocation requirements and resource availability, and thread scheduling is executed.

[0069] Based on node classification, resource allocation requirements, resource availability, and thread scheduling results, a resource allocation requirement report is generated.

[0070] Specifically, load parameters typically include node CPU utilization, memory usage, and other resource consumption data. To calculate the load value for each computing node, a weighted summation method is used, averaging the various load parameters according to preset weights to obtain the total load value for each node. This calculation method ensures that the impact of different resource types is reasonably integrated to reflect the actual load situation of the nodes, and the load value can quantify the resource consumption of each node in the current time period.

[0071] After calculating the load value of each node, the load value is compared with a preset load threshold to determine the node's load status. When a node's load value exceeds the set threshold, the node is classified as a load surge node; otherwise, it is classified as an idle node. This classification is based on the comparison result with the load threshold. A load surge node indicates that the current load is exceeding expectations and requires immediate scheduling of more resources for processing, while an idle node indicates that its resource utilization is low and can be used to receive additional tasks or migrate data.

[0072] For nodes experiencing a surge in load, the resource allocation requirement for that node is calculated based on its current load and parallel processing capacity. This requirement is estimated based on the relationship between the node's processing capacity (e.g., number of cores, parallel processing capacity) and its current load, aiming to ensure the node receives sufficient resources and prevent overload or performance degradation due to resource shortages. For example, the formula for calculating the resource allocation requirement is: In this context, Rd represents the resource allocation requirement, indicating the additional resources (such as computing cores, memory, bandwidth, etc.) that the node needs to allocate; Ln represents the current load value of the node, indicating the current load status of the node, which can usually be represented by a weighted average of indicators such as CPU utilization and memory usage; Cp represents the parallel processing capability data, indicating the processing capability of the node, which usually includes the number of available computing cores and the number of parallel threads; Pmax represents the maximum parallel processing capability of the node, indicating the maximum load value that the node can bear without overloading (e.g., the maximum number of parallel threads that can be processed or the maximum number of computing tasks that can be carried); and Pn represents the current parallel load value of the node, indicating the current amount of parallel tasks of the node, which may be lower than the maximum parallel capability.

[0073] Similarly, for idle nodes, the available resources of the node are calculated based on its remaining computing resources and available bandwidth, and appropriate tasks are assigned to it. In this way, it is possible to ensure that the resources of nodes experiencing load surges and idle nodes are rationally allocated, avoiding waste of system resources or excessive load.

[0074] To ensure data consistency between nodes experiencing load surges and idle nodes, a cache consistency protocol is used to maintain the synchronization of packet data. A cache consistency protocol refers to a mechanism in a multiprocessor system that ensures all processors see the same cached data, such as the MESI protocol, where M stands for Modified, E for Exclusive, S for Shared, and I for Invalid. Data consistency is maintained through state transitions. Specifically, the cache consistency protocol writes packet data to a shared cache and broadcasts update messages to all nodes to ensure that each node can synchronize the latest packet data in a timely manner. If a node receives data with an invalid state, it pulls the latest data from the shared cache. Furthermore, a write invalidation strategy is employed. When a node updates a packet during a load surge, the cache replicas of other nodes become invalid, forcing a reread of the updated data. This mechanism reduces synchronization latency and improves system response speed. In large-scale clusters, the cache consistency protocol uses hierarchical caching and a directory-based consistency protocol to effectively reduce communication overhead and ensure data synchronization integrity under high load scenarios.

[0075] Once data synchronization is complete, the system determines whether the activation conditions for the thread scheduling mechanism are met based on the synchronization results. Specifically, the activation condition for the thread scheduling mechanism is based on the completeness of the synchronization results. The scheduling mechanism is only triggered when the group data consistency rate exceeds a certain threshold (e.g., 95%), in which resource migration and thread scheduling are performed. When the activation conditions are met, the system calculates the priority of thread migration based on the resource requirements of the load-surge nodes and the resource availability of the idle nodes, and executes thread scheduling to migrate tasks from the load-surge nodes to the idle nodes, balancing resource allocation among the nodes. For example, the priority of thread migration can be calculated using the following formula: Here, Tpriority represents the thread migration priority, indicating the priority of each thread migration; threads with higher priority will be scheduled first. Rd represents the resource allocation requirement, indicating the current resource requirement of the node (as calculated in the previous step). Ra represents the resource availability, indicating the remaining available computing resources on the node (including computing cores, memory, bandwidth, etc.). This can be obtained through metrics such as the number of idle computing cores and the capacity of idle memory on the computing node. Wd is a weighting coefficient used to balance the demand for different resources, which is usually adjusted according to actual needs. For example, more weight may be given to compute-intensive tasks, while memory may have a higher weight for memory-intensive tasks.

[0076] Finally, based on node classification, resource allocation requirements, resource availability, and thread scheduling results, a resource allocation requirement report is generated. The report includes the resource allocation status of each node and how to adjust resources to achieve load balancing.

[0077] This technical solution enables efficient resource allocation and scheduling, ensuring that nodes experiencing load surges receive sufficient resource support while optimizing resource utilization of idle nodes. It not only improves system resource utilization but also avoids performance bottlenecks caused by node overload or resource waste, effectively enhancing system stability and response speed.

[0078] Step 3: If the node classification results show that the load value of a certain type of node exceeds the preset threshold, the resource adjustment mechanism is triggered. The node resources are remapped through resource virtualization technology to obtain the adjusted node resource configuration.

[0079] In one specific embodiment, the process of performing step 3 may specifically include the following steps:

[0080] For nodes experiencing load surges, their load values ​​are determined based on real-time monitoring results. It is then determined whether the load value exceeds a preset threshold. If so, the node experiencing a load surge is confirmed to be in an overloaded state.

[0081] Upon detecting an overloaded node, an in-situ resource adjustment mechanism is triggered. While ensuring uninterrupted node operation, the system enters the resource virtualization layer to perform a resource remapping operation. The resource remapping operation involves identifying the current resource allocation status of each computing node, mapping a portion of the load from the overloaded node to an idle node based on a load balancing algorithm, and updating the virtual resource pool.

[0082] Based on the results of the remapping operation, the adjusted node resource configuration is generated;

[0083] The adjusted node resource configuration is encoded into a remapping signal via the interconnect bus and transmitted using the bus protocol;

[0084] Based on the remapping signal, the security encryption module is triggered to perform integrity verification on the adjusted node resource configuration;

[0085] If the verification passes, the adjusted node resource configuration is confirmed to be valid, and a resource configuration report containing details of the adjusted resource configuration and the verification results is generated.

[0086] Specifically, during real-time monitoring, the agent module continuously monitors the load data of the computing nodes and compares it with a preset load threshold. When the load value exceeds the threshold, it confirms that the node is overloaded and then initiates an in-situ resource adjustment mechanism.

[0087] When a node is identified as experiencing a load surge, a resource remapping operation is performed through the resource virtualization layer. The resource virtualization layer acts as a software abstraction layer, enabling the conversion of physical resources into virtual resources and dynamic adjustments without affecting node operation. Specifically, the current resource allocation of the node experiencing the load surge is first identified. For example, resource information such as CPU utilization and memory allocation is obtained through the agent module to determine if the node's resource usage is nearing its limit. Next, based on load balancing algorithms (such as round-robin or weighted allocation algorithms), some tasks are transferred from the node experiencing the load surge to idle nodes to achieve load balancing. The remapping operation updates the virtual resource pool, ensuring that running tasks or applications are not affected during the allocation process.

[0088] After the remapping operation, an adjusted node resource configuration is generated. This configuration reflects the new resource allocation, indicating that nodes with high loads have been optimized to reduce their burden. To ensure the effectiveness and security of the resource configuration, the system transmits the remapping signal to the relevant nodes through an interconnect bus design. The interconnect bus design is a communication architecture designed to achieve high-speed data transmission and low latency. Through this architecture, the adjusted resource configuration can be promptly transmitted to all relevant nodes. During the transmission of the remapping signal, data is encapsulated and transmitted using a bus protocol (such as PCIe), ensuring no data loss or corruption during transmission and maintaining low latency. To ensure data consistency and integrity during transmission, the system employs a cache consistency protocol to maintain data synchronization between nodes experiencing surges in load and idle nodes. For example, during data transmission, the MESI protocol (Modify, Exclusive, Shared, Invalid State Protocol) is used to guarantee the consistency of cached data, avoiding system errors caused by inconsistencies in cached data across different nodes.

[0089] Subsequently, a secure encryption module is activated based on the remapping signal to verify the integrity of the adjusted node resource configuration. This secure encryption module uses the AES algorithm to encrypt the remapping signal, generates a hash value, and compares it with a pre-stored baseline hash value. If the comparison result matches, the integrity of the resource configuration is confirmed, and a verification report is generated. This report includes details of the adjusted resource configuration, the verification results, and any anomaly records, ensuring the security and reliability of the configuration.

[0090] Through the above steps, load balancing can be effectively achieved under conditions of surging node load using resource virtualization technology and cache consistency protocols, while ensuring the integrity of node resource configuration. This process effectively avoids problems such as system overload and performance degradation, improving the utilization of computing resources and system stability. For high-concurrency processing scenarios, especially in multi-stream and big data processing tasks, resource remapping and optimization can significantly improve system response speed and processing capacity, meeting the needs of efficient resource scheduling and dynamic adjustment.

[0091] The implementation of this technology solves the problem of how to optimize resource allocation through resource virtualization when the computing node load is uneven, improves the system load adaptability and overall performance, and thus effectively enhances the system's stability and scalability.

[0092] In a preferred embodiment, triggering the security encryption module to perform integrity verification on the adjusted node resource configuration based on the remapping signal includes:

[0093] Based on the remapping signals from the interconnect bus, the verification requirements for the adjusted node resource configuration are determined.

[0094] The secure encryption module generates an encrypted verification key based on the node identifier and resource parameters in the remapping signal.

[0095] Using the encrypted verification key, perform encrypted hash calculation on the grouped data of the adjusted node resource configuration, compare the calculation result with the pre-stored benchmark, and determine the integrity status of the resource configuration based on the comparison result;

[0096] If the integrity status is abnormal, the retransmission of the remapping signal is triggered, and a new remapping signal is sent through the backup path;

[0097] The resource configuration of the computing nodes is updated by retransmitting the remapping signal;

[0098] Based on the updated resource configuration and verification process, generate an integrity verification report;

[0099] Based on the abnormal frequency and transmission delay in the verification report, the optimization parameters of the security encryption module were adjusted, including the key generation cycle and key length.

[0100] Specifically, the use of the security encryption module is a crucial step in ensuring the integrity and security of node resource configuration. Based on the remapping signal from the interconnect bus, the verification requirements for the adjusted node resource configuration are determined. This signal carries the node identifier and resource parameters, which provide information about the adjusted resource status. The security encryption module first extracts the node identifier and resource configuration data from the signal; this data is used to generate an encryption verification key. This key is generated using a symmetric encryption algorithm (such as AES), with the node identifier and resource parameters as input sources. A pseudo-random function is used to generate the final key, ensuring that the generated key possesses sufficient security and unpredictability, meeting the system's data protection requirements.

[0101] After generating the encryption verification key, the security encryption module applies this key to perform a cryptographic hash calculation on each packet of data in the adjusted node resource configuration. Cryptographic hashing is a common security algorithm that ensures the resource configuration data is not tampered with during transmission by converting it into a fixed-length hash value. This calculated hash value is compared with a pre-stored baseline hash value. If they match, it indicates that the data has maintained its integrity during transmission. If the hash values ​​match, the resource configuration is marked as "intact," meaning it has not been tampered with; if the match fails, it is marked as "abnormal," indicating that the resource configuration may have been modified or that data loss or corruption occurred during transmission.

[0102] During integrity verification, if the verification result indicates an anomaly in resource configuration, a retransmission of the remapping signal is triggered. At this point, thanks to the interconnect bus design, the signal is transmitted via redundant paths, ensuring its reliability and integrity. During transmission, the system design employs multi-path redundancy technology to prevent single points of failure, ensuring that data reaches the target node promptly without data loss. This mechanism is particularly important for high-load and large-scale cluster environments because it effectively avoids data synchronization failures due to a single network channel failure, thereby improving the overall fault tolerance of the system.

[0103] After the remapping signal is retransmitted, the node's resource configuration is updated, and the load is adjusted according to the new configuration to ensure resource balance among nodes. After the resource configuration update is complete, an integrity verification report is generated. This report records all configuration adjustments, the verification process, and the verification results. This report includes not only whether the integrity verification passed or failed, but also important parameters such as anomaly frequency and transmission latency. This information provides a basis for subsequent optimization and system maintenance, helping system administrators understand potential problems that may exist during resource allocation.

[0104] Based on the anomaly frequency and transmission latency recorded in the verification report, the security encryption module can optimize its parameter configuration. For example, by analyzing the transmission latency, it can be determined whether to adjust the generation cycle of the encryption verification key to improve verification efficiency and system response speed. In some high-load scenarios, the key generation cycle can be optimized from the default 10 seconds to a shorter cycle, such as 5 seconds, to reduce latency in the verification process and improve the system's real-time response capability. Furthermore, the key length can also be dynamically adjusted, using a longer key (e.g., 256 bits) when the system load is high to enhance data security.

[0105] By using a secure encryption module to verify the integrity of adjusted node resource configurations, unauthorized tampering during dynamic adjustments can be effectively prevented, ensuring the accuracy and security of system resource allocation. When dealing with large-scale clusters, this method enhances system fault tolerance, reduces resource configuration problems caused by network failures or attacks, and improves system stability and reliability. Therefore, this technical solution effectively solves the problem of ensuring resource configuration integrity and security in a distributed computing environment, ensuring the accuracy and efficiency of resource scheduling between nodes.

[0106] Step 4: Build a new cluster structure based on the adjusted node resource configuration, extract the reconstruction parameters, and determine the resource reconstruction scheme based on the reconstruction parameters.

[0107] In one specific embodiment, the process of performing step 4 may specifically include the following steps:

[0108] Based on the adjusted node resource configuration, determine the cluster structure parameters, including the number of nodes, load distribution, and connection topology;

[0109] The cluster structure parameters are classified and processed by a dynamic scheduling engine, and a priority queue algorithm is used to prioritize the processing of high-load parameters.

[0110] Based on the processed parameter set, a highly available cluster structure with redundant node configuration is generated.

[0111] Key performance metrics are extracted from the high-availability cluster structure as reconstruction parameters, including node utilization and data flow rate.

[0112] When the node utilization exceeds the preset utilization threshold, a load balancing algorithm is applied based on the reconstruction parameters to determine a resource reconstruction scheme, which includes the task migration target.

[0113] By using a dynamic scheduling engine, the execution path of the resource reconfiguration scheme is optimized using the shortest path algorithm, and network latency bottlenecks are identified and avoided.

[0114] Based on the optimized execution path, a localized data storage path is generated, and the availability of the cluster structure is verified through simulated data access tests.

[0115] Generate a configuration report that includes the cluster structure, resource refactoring scheme, optimized execution path, and verification results.

[0116] Specifically, based on the adjusted node resource configuration, key cluster parameters are determined. These parameters include the number of compute nodes, load distribution, and cluster connection topology. Subsequently, the cluster structure parameters are further processed by a dynamic scheduling engine. This engine employs a real-time feedback mechanism, continuously and iteratively checking the cluster status and dynamically adjusting resource allocation to adapt to constantly changing load demands. By categorizing cluster parameters, the dynamic scheduling engine can effectively identify static parameters, such as the number of nodes and network topology, and process them separately from dynamic parameters, such as load changes. This process improves scheduling efficiency and ensures sufficient flexibility for the system when handling large data streams or multi-threaded parallel tasks.

[0117] After classification, the dynamic scheduling engine uses a priority queue algorithm to sort the dynamic components, ensuring that high-load tasks are processed first. This method effectively improves resource allocation efficiency when handling high-load nodes, avoids excessive resource consumption concentrated on certain nodes, and ensures the balance and stability of the entire cluster. In this way, the dynamic scheduling engine not only optimizes the use of cluster resources but also lays the foundation for the subsequent generation of a high-availability cluster.

[0118] A high-availability cluster architecture refers to a cluster architecture configured with redundancy and fault tolerance. Its purpose is to ensure that the system can continue to operate normally and provide uninterrupted service even in the event of hardware failures, network latency, or other unforeseen events. The main characteristics of a high-availability cluster are redundancy, fault tolerance, and automatic recovery capabilities. The process of generating a high-availability cluster architecture with redundant node configurations can be divided into several stages: 1. Analyzing the processed cluster architecture parameters: First, it is necessary to identify the nodes with high loads, especially when the computing power or storage resources of some nodes are close to saturation. By processing the load distribution data, it is possible to identify which nodes are at risk of overload and thus predict which nodes may become system bottlenecks; 2. Introducing redundant node configurations: Once high-load nodes are identified, the cluster architecture will automatically adjust resource configurations based on load distribution and network topology, adding redundant nodes. Redundant nodes are extra nodes reserved in the cluster. These nodes are not necessarily used 24 / 7, but they can be activated at any time to take over the tasks of high-load nodes. These redundant nodes, through appropriate configuration and resource mapping mechanisms, can automatically take over the computing tasks of overloaded nodes when the load surges, thereby effectively distributing system pressure and ensuring continuous system operation. 3. Combination of load balancing and redundant nodes: Through load balancing algorithms, redundant nodes not only serve as system backups but also participate in load distribution. The algorithm selects the most suitable redundant nodes based on the current load distribution and task priorities, migrating some tasks to these nodes to prevent single points of failure or overload. The dynamic scheduling engine adjusts the participation of redundant nodes based on these real-time changes. For example, in a large-scale domestic cloud computing environment, the cluster has multiple computing nodes distributed across different geographical locations, each undertaking different types of tasks, such as data storage, computing, or task scheduling. At a certain moment, some nodes in the cluster experience a sharp increase in load due to overload or hardware failure. At this time, the dynamic scheduling engine, based on real-time monitoring results, identifies the nodes with high loads and reallocates resources through the configuration of redundant nodes. Under the guidance of the load balancing algorithm, these redundant nodes begin to take over some tasks, preventing further pressure on overloaded nodes while ensuring the performance and stability of the entire cluster.

[0119] After generating a high-availability cluster structure, key performance indicators are extracted from it as reconstruction parameters for resource refactoring. These parameters include critical factors such as node utilization and data flow rate, which directly impact the overall cluster performance and load balancing. Node utilization, as a core metric for measuring node load, reflects whether a node is overloaded and whether task migration or resource expansion is necessary. When node utilization exceeds a preset threshold, a resource refactoring scheme is determined using a load balancing algorithm. Load balancing algorithms are typically based on hashing methods to evenly distribute tasks across different computing nodes, preventing some nodes from being overloaded while others are idle. This algorithm effectively distributes load, ensuring balanced use of cluster resources and reducing performance bottlenecks caused by uneven load distribution.

[0120] The execution path of the resource reconfiguration scheme also needs optimization. Through a dynamic scheduling engine, the execution path is optimized using a shortest path algorithm. Shortest path algorithms, such as Dijkstra's algorithm, can calculate the minimum cost path, thereby avoiding network latency and bottlenecks and ensuring the efficient flow of data within the cluster. By iteratively calculating path costs, the system can continuously optimize the execution path, reduce transmission latency, and improve system response speed. After optimizing the execution path, a data-local storage path is generated. The data-local storage path maps storage locations according to the optimized path, ensuring that data is stored on the most suitable node, thereby improving data access speed and reducing latency caused by cross-node access. Finally, simulated data access tests are used to verify the availability of the cluster structure, ensuring that the data-local storage path can effectively support data access needs and guarantee the high availability of the system.

[0121] Throughout the process, the generated resource refactoring configuration report summarizes the cluster structure, resource refactoring scheme, optimized execution path, and verification results. This report provides system administrators with a clear record of configuration adjustments and performance verification results, which helps in further resource optimization and troubleshooting.

[0122] This technical solution can flexibly adjust the cluster structure based on real-time monitoring and load data, and improve resource utilization efficiency through optimized algorithms, ensuring the stability and scalability of the system under high load and multi-tasking scenarios. It effectively solves the problems of resource imbalance and system overload, improves the utilization rate of cluster resources, optimizes data processing paths, and enhances the overall performance of the cluster.

[0123] Step 5: Execute the data migration process according to the resource reconstruction plan, generate the migration path using the compatibility interface algorithm, and determine the state synchronization integrity during the migration process based on the migration path.

[0124] In one specific embodiment, the process of performing step 5 may specifically include the following steps:

[0125] Based on the resource reconfiguration scheme, the node load data and available resource indicators are analyzed to determine the source and target nodes for data migration.

[0126] The compatibility interface algorithm is used to evaluate the compatibility of interface protocols and data formats between the source node and the target node, and the shortest path algorithm is used to generate the data migration path between the source node and the target node based on the evaluation results.

[0127] Based on the data migration path, execute the segmented transmission and confirmation process for data blocks;

[0128] During the migration process, the power consumption data of each node is monitored in real time through power management strategies, and power consumption values ​​are collected and energy consumption sequences are recorded.

[0129] Based on energy consumption data, time series analysis is used to identify abnormal energy consumption peaks and determine the intervention points for fault isolation technology.

[0130] When an intervention point is triggered, fault isolation technology is used to switch to an alternative path or isolate abnormal nodes to restore the normal migration process.

[0131] Based on the data migration path, a hash algorithm is used to compare the data integrity between the source node and the target node to verify whether the state synchronization is complete. If not, a retransmission mechanism is triggered to re-execute the data migration.

[0132] Generate a data migration execution report that includes migration duration, total energy consumption, and synchronization verification results.

[0133] Specifically, based on the resource reconfiguration plan, the system analyzes node load data and available resource metrics. The purpose of this process is to determine which computing nodes are overloaded and require data migration, and which nodes have sufficient resources and are suitable as target nodes for data migration. Specifically, node load data is obtained through real-time monitoring, including CPU utilization and memory usage. When calculating available resource metrics, the remaining computing power and storage space of each node are considered. This data helps the system identify overloaded nodes and lightly loaded target nodes, thus preparing for the migration process.

[0134] During the data migration path generation process, a compatibility interface algorithm is used to ensure data format and protocol compatibility between the source and target nodes. This process first evaluates the interface protocol versions and data formats of the source and target nodes. If the interfaces are compatible, the next step is to use a shortest path algorithm (such as Dijkstra's algorithm) to calculate the optimal migration path from the source to the target node. In this process, the path weights are primarily determined based on network transmission latency and bandwidth. The shortest path algorithm helps the system find the optimal data migration path in complex network topologies, reducing latency and packet loss during data transmission and ensuring efficient data migration.

[0135] Based on the generated data migration path, a segmented transmission and acknowledgment process for data blocks will be executed. In this stage, data is divided into multiple small blocks for transmission according to the migration path, and acknowledgment is performed after each data block transmission is completed. This segmented transmission method ensures the accuracy of data migration and avoids total failure due to a single point of failure during large-scale data transmission.

[0136] During data migration, the power management strategy monitors the energy consumption data of each node in real time. The strategy reads the voltage and current of each node using sensors and calculates the power consumption of each node. This data helps the system detect potential energy consumption anomalies during migration, such as excessive power consumption. The system processes this abnormal data through time series analysis to identify energy consumption peaks and determine intervention points for fault isolation technology. An intervention point refers to the moment or location during data migration when the system detects an abnormal energy consumption peak, triggering the intervention of fault isolation technology. Specifically, the intervention point is the critical moment when the system determines to take corrective measures on nodes with potential failures or performance bottlenecks. These measures may include switching to a backup path, isolating faulty nodes, or adjusting resource allocation to ensure the stability of the entire system and the normal progress of data migration. Through this process, the system can effectively prevent migration failures caused by overload or hardware failures and take timely measures to avoid system damage.

[0137] When an intervention point is triggered, fault isolation technology switches to an alternative path or removes the abnormal node from the migration path, thereby restoring the normal migration process. This technology ensures that the migration process is not affected for an extended period in the event of an anomaly, allowing the system to continue operating and guaranteeing the smooth progress of data migration.

[0138] In the final stage of data migration, the system uses a hash algorithm (such as SHA-256) to verify the data integrity of the source and target nodes. By calculating the hash value of the data and comparing it with the hash value of the data on the target node, the system ensures that the data has not been tampered with during the migration process. If the hash values ​​match, the system confirms that the state synchronization of the migration process is complete; if they do not match, a retransmission mechanism is triggered, and the data migration is re-executed. In this way, through hash algorithm verification, the system can guarantee the security and consistency of data, avoiding failures caused by data corruption or loss.

[0139] Finally, the system generates a data migration execution report, which includes information such as migration time, total energy consumption, and synchronization verification results. This report helps system administrators understand the overall data migration process and provides feedback for subsequent resource refactoring plans.

[0140] This technical solution organically combines a series of techniques, including compatibility interface algorithms, shortest path algorithms, power management strategies, fault isolation techniques, and data integrity verification, to ensure the efficiency, security, and stability of the data migration process. Especially in cluster environments, this solution can monitor any potential problems during the data migration process in real time and respond quickly, guaranteeing high system availability.

[0141] Step 6: If the state synchronization integrity is confirmed, the dynamic management strategy is activated and a resource allocation model is generated, thereby achieving stable allocation of system resources and overall operational stability.

[0142] In one specific embodiment, the process of performing step 6 may specifically include the following steps:

[0143] When the verification result of state synchronization integrity is complete, the dynamic management strategy is activated;

[0144] Using independently controllable software, a genetic algorithm is employed to iteratively optimize the execution parameters of the dynamic management strategy. These parameters include load threshold, resource allocation ratio, and synchronization interval.

[0145] Based on the optimized execution parameters, a resource allocation model describing the resource mapping relationship between nodes is constructed;

[0146] By using a thread scheduling mechanism and applying a priority queue algorithm, the optimization results of the resource allocation model are integrated to form a unified optimization dataset.

[0147] Based on the optimized dataset, node group data is extracted, and the parallel processing capability index of each computing node is calculated. The k-means clustering algorithm is used to analyze the node group data and the parallel processing capability index to determine the balance index of parallel processing capability.

[0148] Based on the balance index, the operation scenario of the resource allocation model is simulated, the response time and error rate of the resource allocation model under different load conditions are calculated, and it is verified whether the resource allocation model meets the preset stability threshold.

[0149] If the stability verification fails, adjust the parameters of the resource allocation model and regenerate the model until the verification passes.

[0150] Generate a resource allocation model report that includes the final resource allocation model, balance index, and stability verification results.

[0151] Specifically, it is necessary to ensure that the state synchronization integrity is verified as complete, guaranteeing that the data and resource configurations of all nodes in the system are fully synchronized. Based on this verification result, dynamic management strategies will be activated to optimize resource allocation and improve the overall system efficiency and stability.

[0152] In optimizing the resource allocation model, a genetic algorithm is used in independently controllable software to iteratively optimize the execution parameters of the dynamic management strategy. The genetic algorithm is an optimization method that simulates natural selection, iteratively optimizing parameters through mechanisms such as selection, crossover, and mutation to ultimately obtain the optimal solution. In this process, execution parameters include load thresholds, resource allocation ratios, and synchronization intervals, which play a crucial role in balancing the system's resource scheduling and processing capacity. For example, by adjusting the load threshold from 0.8 to 0.75, the system can better adapt to resource imbalances that occur during real-time monitoring. The optimized execution parameters ensure the system's response speed and efficient resource utilization.

[0153] By optimizing the execution parameters, a resource allocation model describing the resource mapping relationship between nodes is constructed. After the resource allocation model is completed, the thread scheduling mechanism applies a priority queue algorithm to integrate the optimization results of the resource allocation model, forming a unified optimized dataset. The priority queue algorithm is a data structure for managing priority tasks, ensuring that high-priority tasks are processed first, which is particularly important for nodes with high resource requirements. In the cluster, node load processing is optimized using this algorithm, ensuring that heavily loaded nodes are processed first, thereby improving overall resource utilization and efficiency. The optimized dataset refers to the collection formed by the optimized resource allocation model after integration through the thread scheduling mechanism and the priority queue algorithm. This dataset contains key information related to node resource allocation and processing capabilities, supporting subsequent resource scheduling and load balancing in the system. The optimized dataset includes, but is not limited to, the following aspects: node configuration data (optimized node resource configuration, such as the allocation of resources (e.g., CPU, memory, bandwidth) for each node), load distribution data (optimized node load distribution, typically including the load value of each node (e.g., CPU utilization, memory utilization), task priority data (task priority data generated through a priority queue algorithm), resource mapping relationships (resource mapping relationships between nodes in the resource allocation model, reflecting the association and interdependence of resources between nodes), parallel processing capability data (parallel processing capability indicators for each node, including how many tasks each node can process in parallel, throughput, etc.), resource demand and availability data (data comparing the resource demand of each node with the current available resources), and scheduling decision support data (recommendations for resource allocation and scheduling decisions generated based on the above data).

[0154] Based on the optimized dataset, node grouping data is extracted, and the parallel processing capability index of each computing node is calculated. Then, the k-means clustering algorithm is used to analyze the node grouping data and parallel processing capability to determine the balance index of parallel processing capability. The k-means clustering algorithm divides nodes into multiple groups by iteratively calculating the distance from data points to cluster centers, ensuring that the processing capability within each group is balanced and avoiding overload of some nodes, thus affecting the overall system performance. For example, the formula for calculating the balance index is: Where k is the number of clusters, usually k=2, indicating that the nodes are divided into two categories: high load and low load. SSEi is the sum of squared errors in the i-th cluster, N is the total number of nodes, and Pj is the parallel processing capability index of the j-th node.

[0155] By using a balance index, the system simulates the operational scenarios of the resource allocation model, calculating its response time and error rate under different load conditions. This step verifies the model's stability under dynamic loads by simulating real-world scenarios, ensuring stable operation across various workloads. If the model fails the stability verification, the system automatically adjusts the parameters of the resource allocation model and regenerates it until it meets the preset stability threshold. Through this iterative optimization, the system ensures the efficiency and stability of the resource allocation model.

[0156] Finally, a report is generated that includes the optimized resource allocation model, balance metrics, and stability verification results. This report summarizes the key indicators in the resource allocation process. Using this method, the system can adjust the resource allocation model in real time, improving overall system performance and resource utilization while ensuring system stability.

[0157] This technical solution effectively addresses issues such as uneven resource allocation and system instability in complex scenarios involving high loads and multi-stream data processing by activating dynamic management strategies and continuously optimizing the resource allocation model. Through optimized execution parameters and cluster structure configuration, it can respond to changes in resource demands in real time and maintain efficient and stable operation under various load conditions, significantly improving resource utilization and cluster stability.

[0158] The above describes the resource scheduling method for domestically developed cloud computing based on a multi-core architecture in the embodiments of this application. The following describes the resource scheduling system for domestically developed cloud computing based on a multi-core architecture in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 2 This application provides a schematic diagram of an embodiment of a cloud resource scheduling system based on a multi-core architecture. The system includes:

[0159] The data acquisition module 10 is used to collect load data and chip integration information of each node through the agent module deployed on the computing node, generate real-time monitoring results, and determine the resource distribution between nodes.

[0160] The group evaluation module 20 is used to group computing nodes according to real-time monitoring results using a load balancing algorithm, obtain node classification results, and determine the resource allocation requirements of each node.

[0161] The resource adjustment module 30 is used to trigger a resource adjustment mechanism when the load value of a certain type of node exceeds a preset threshold, based on the node classification results. The mechanism uses resource virtualization technology to remap the node resources and obtain the adjusted node resource configuration.

[0162] The structure building module 40 is used to build a new cluster structure based on the adjusted node resource configuration, extract reconstruction parameters, and determine a resource reconstruction scheme based on the reconstruction parameters.

[0163] The migration execution module 50 is used to execute the data migration process according to the resource reconstruction scheme, generate the migration path using the compatibility interface algorithm, and determine the state synchronization integrity during the migration process based on the migration path.

[0164] The strategy activation module 60 is used to activate the dynamic management strategy and generate a resource allocation model when the integrity of state synchronization is confirmed, thereby achieving stable allocation of system resources and overall operational stability.

[0165] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A resource scheduling method for domestically developed cloud computing based on a multi-core architecture, characterized in that: The method includes: Step 1: By using the agent module deployed on the computing nodes, collect the load data and chip integration information of each node, generate real-time monitoring results, and determine the resource distribution among the nodes; Step 2: Based on the real-time monitoring results, apply the load balancing algorithm to group the computing nodes, obtain the node classification results, and determine the resource allocation requirements of each node. Step 3: If the node classification results show that the load value of a certain type of node exceeds the preset threshold, the resource adjustment mechanism is triggered. The node resources are remapped through resource virtualization technology to obtain the adjusted node resource configuration. Step 4: Construct a new cluster structure based on the adjusted node resource configuration, extract reconstruction parameters, and determine a resource reconstruction scheme based on the reconstruction parameters; Step 5: Execute the data migration process according to the resource restructuring plan, generate the migration path using the compatibility interface algorithm, and determine the integrity of the state synchronization during the migration process based on the migration path; Step 6: If the integrity of state synchronization is confirmed, the dynamic management strategy is activated and a resource allocation model is generated, thereby achieving stable allocation of system resources and overall operational stability. The node classification results include nodes experiencing load surges and idle nodes. Step 3 includes: For nodes experiencing load surges, their load values ​​are determined based on real-time monitoring results. It is then determined whether the load value exceeds a preset threshold. If so, the node experiencing a load surge is confirmed to be in an overloaded state. Upon detecting an overloaded node, an in-situ resource adjustment mechanism is triggered. While ensuring uninterrupted node operation, the system enters the resource virtualization layer to perform a resource remapping operation. The resource remapping operation involves identifying the current resource allocation status of each computing node, mapping a portion of the load from the overloaded node to an idle node based on a load balancing algorithm, and updating the virtual resource pool. Based on the results of the remapping operation, the adjusted node resource configuration is generated; The adjusted node resource configuration is encoded into a remapping signal via the interconnect bus and transmitted using the bus protocol; Based on the remapping signal, the security encryption module is triggered to perform integrity verification on the adjusted node resource configuration; If the verification passes, the adjusted node resource configuration is confirmed to be valid, and a resource configuration report containing details of the adjusted resource configuration and the verification results is generated.

2. The resource scheduling method for domestically developed cloud computing based on a multi-core architecture as described in claim 1, characterized in that, Step 1 includes: By using agent modules deployed on each computing node, load data and chip integration information of the computing node are periodically collected through a polling mechanism. Based on load data, a weighted average method is used to calculate the resource utilization of each computing node; Based on chip integration information, the parallel processing capability data of each computing node is analyzed and extracted; By integrating resource utilization and parallel processing capability data, structured real-time monitoring results are generated. Based on real-time monitoring results, the variance or standard deviation of resource utilization rates among nodes is calculated to determine whether there is an uneven distribution of resources among the nodes. If it is determined that there is an uneven distribution of resources, the scalability configuration requirements are determined based on the degree of unevenness and historical trends. The scalability configuration requirements include the number of new nodes, virtual resource mapping strategies and / or parameters for improving parallel capabilities. Based on parallel processing capabilities, a multi-threaded parallel algorithm is used to process multiple monitoring data streams and summarize them to generate an analysis report that includes resource distribution statistics and optimization suggestions.

3. The resource scheduling method for domestically developed cloud computing based on a multi-core architecture as described in claim 2, characterized in that, Step 2 includes: The load parameters of each computing node are extracted based on the real-time monitoring results, and the load parameters of each computing node are weighted and summed to obtain the load value of each computing node. The load value is compared with a preset load threshold, and the computing nodes are classified into load surge nodes and idle nodes based on the comparison results. For nodes experiencing a surge in load, the resource allocation requirement is calculated based on the node's current load value and parallel processing capacity data. For idle nodes, the amount of available resources is determined based on their remaining computing resources and available bandwidth; The cache consistency protocol maintains the synchronization of packet data between nodes experiencing load surges and idle nodes, including writing packet data to a shared cache, broadcasting update messages, and employing a write failure strategy to ensure data consistency. Based on the consistency rate of grouped data synchronization, determine whether the activation conditions of the thread scheduling mechanism are met; When the activation conditions are met, the thread migration priority is calculated based on the resource allocation requirements and resource availability, and thread scheduling is executed. Based on node classification, resource allocation requirements, resource availability, and thread scheduling results, a resource allocation requirement report is generated.

4. The resource scheduling method for domestically developed cloud computing based on a multi-core architecture as described in claim 1, characterized in that, Step 4 includes: Based on the adjusted node resource configuration, determine the cluster structure parameters, including the number of nodes, load distribution, and connection topology; The cluster structure parameters are classified and processed by a dynamic scheduling engine, and a priority queue algorithm is used to prioritize the processing of high-load parameters. Based on the processed parameter set, a highly available cluster structure with redundant node configuration is generated. Key performance metrics are extracted from the high-availability cluster structure as reconstruction parameters, including node utilization and data flow rate. When the node utilization exceeds the preset utilization threshold, a load balancing algorithm is applied based on the reconstruction parameters to determine a resource reconstruction scheme, which includes the task migration target. By using a dynamic scheduling engine, the execution path of the resource reconfiguration scheme is optimized using the shortest path algorithm, and network latency bottlenecks are identified and avoided. Based on the optimized execution path, a localized data storage path is generated, and the availability of the cluster structure is verified through simulated data access tests. Generate a configuration report that includes the cluster structure, resource refactoring scheme, optimized execution path, and verification results.

5. The resource scheduling method for domestically developed cloud computing based on a multi-core architecture as described in claim 1, characterized in that, Step 5 includes: Based on the resource reconfiguration scheme, the node load data and available resource indicators are analyzed to determine the source and target nodes for data migration. The compatibility interface algorithm is used to evaluate the compatibility of interface protocols and data formats between the source node and the target node, and the shortest path algorithm is used to generate the data migration path between the source node and the target node based on the evaluation results. Based on the data migration path, execute the segmented transmission and confirmation process for data blocks; During the migration process, the power consumption data of each node is monitored in real time through power management strategies, and power consumption values ​​are collected and energy consumption sequences are recorded. Based on energy consumption data, time series analysis is used to identify abnormal energy consumption peaks and determine the intervention points for fault isolation technology. When an intervention point is triggered, fault isolation technology is used to switch to an alternative path or isolate abnormal nodes to restore the normal migration process. Based on the data migration path, a hash algorithm is used to compare the data integrity between the source node and the target node to verify whether the state synchronization is complete. If not, a retransmission mechanism is triggered to re-execute the data migration. Generate a data migration execution report that includes migration duration, total energy consumption, and synchronization verification results.

6. The resource scheduling method for domestically developed cloud computing based on a multi-core architecture according to claim 1, characterized in that, Step 6 includes: When the verification result of state synchronization integrity is complete, the dynamic management strategy is activated; Using independently controllable software, a genetic algorithm is employed to iteratively optimize the execution parameters of the dynamic management strategy. These parameters include load threshold, resource allocation ratio, and synchronization interval. Based on the optimized execution parameters, a resource allocation model describing the resource mapping relationship between nodes is constructed; By using a thread scheduling mechanism and applying a priority queue algorithm, the optimization results of the resource allocation model are integrated to form a unified optimization dataset. Based on the optimized dataset, node group data is extracted, and the parallel processing capability index of each computing node is calculated. The k-means clustering algorithm is used to analyze the node group data and the parallel processing capability index to determine the balance index of parallel processing capability. Based on the balance index, the operation scenario of the resource allocation model is simulated, the response time and error rate of the resource allocation model under different load conditions are calculated, and it is verified whether the resource allocation model meets the preset stability threshold. If the stability verification fails, adjust the parameters of the resource allocation model and regenerate the model until the verification passes. Generate a resource allocation model report that includes the final resource allocation model, balance index, and stability verification results.

7. The resource scheduling method for domestically developed cloud computing based on a multi-core architecture as described in claim 2, characterized in that, The data, based on parallel processing capabilities, employs a multi-threaded parallel algorithm to process multiple monitoring data streams, and summarizes and generates an analysis report containing resource distribution statistics and optimization suggestions, including: For multiple monitoring data streams, the processing priority of each data stream is calculated based on the data stream size and arrival time, and a priority queue is formed. Retrieve data streams from the priority queue, query available computing core resources, and distribute the core resources evenly to each data stream; Based on the resource allocation results, perform parallel processing of multiple data streams, record the core usage time and memory consumption data of each data stream, and generate a resource distribution data table for each node based on the recorded execution results. Based on the resource distribution data table, calculate the standard deviation of core usage time and memory usage among nodes, and obtain a comprehensive balance score based on a weighted average. If the overall balance score exceeds the preset balance threshold, resource distribution optimization suggestions are generated. The optimization suggestions include the source node to migrate, the target node to migrate, and the migration data stream identifier. Based on the optimization suggestions, the processing strategy for multiple data streams was adjusted, and the priority queue and data stream allocation were updated. The report summarizes the adjusted resource distribution data, overall balance score, optimization suggestions, and the results of implementing the optimization suggestions, and generates an analysis report.

8. The resource scheduling method for domestically developed cloud computing based on a multi-core architecture according to claim 1, characterized in that, The step of triggering the security encryption module to perform integrity verification on the adjusted node resource configuration based on the remapping signal includes: Based on the remapping signals from the interconnect bus, the verification requirements for the adjusted node resource configuration are determined. The secure encryption module generates an encrypted verification key based on the node identifier and resource parameters in the remapping signal. Using the encrypted verification key, perform encrypted hash calculation on the grouped data of the adjusted node resource configuration, compare the calculation result with the pre-stored benchmark, and determine the integrity status of the resource configuration based on the comparison result; If the integrity status is abnormal, the retransmission of the remapping signal is triggered, and a new remapping signal is sent through the backup path; The resource configuration of the computing nodes is updated by retransmitting the remapping signal; Based on the updated resource configuration and verification process, generate an integrity verification report; Based on the abnormal frequency and transmission delay in the verification report, the optimization parameters of the security encryption module were adjusted, including the key generation cycle and key length.

9. A cloud resource scheduling system based on a multi-core architecture, used to implement the cloud resource scheduling method based on a multi-core architecture as described in any one of claims 1 to 8, characterized in that, The system includes: The data acquisition module is used to collect load data and chip integration information of each node through the agent module deployed on the computing node, generate real-time monitoring results, and determine the resource distribution between nodes. The grouping evaluation module is used to group computing nodes based on real-time monitoring results using a load balancing algorithm, obtain node classification results, and determine the resource allocation requirements of each node. The resource adjustment module is used to trigger a resource adjustment mechanism when the load value of a certain type of node exceeds a preset threshold, based on the node classification results. The module uses resource virtualization technology to remap the node resources and obtain the adjusted node resource configuration. The structure building module is used to build a new cluster structure based on the adjusted node resource configuration, extract reconstruction parameters, and determine the resource reconstruction scheme based on the reconstruction parameters. The migration execution module is used to execute the data migration process according to the resource reconstruction plan, generate the migration path using a compatibility interface algorithm, and determine the integrity of the state synchronization during the migration process based on the migration path. The strategy activation module is used to activate dynamic management strategies and generate resource allocation models when the integrity of state synchronization is confirmed, thereby achieving stable allocation of system resources and overall operational stability.

Images