A computing power optimization method and system for cloud computing data center transmission

Abstract

The application is suitable for the field of cloud computing data technology, and provides a computing power optimization method and system for cloud computing data center transmission, which comprises the following steps: adopting a three-level cache mechanism, caching enterprise bank data, and preprocessing the enterprise bank data; based on a computing resource index, a transmission path index and a task dynamic index, a deep scheduling model is constructed; according to business requirements, the enterprise bank data after preprocessing is combined, the deep scheduling model is used to dynamically adjust the computing power allocation and task scheduling in the cloud computing data. The application can fully utilize the computing power resources of the cloud computing data center, avoid waste of resources, improve the resource utilization rate, and reduce the operating cost of the bank.

Description

Technical Field

[0001] This application belongs to the field of cloud computing data technology, and in particular relates to a computing power optimization method and system for cloud computing data center transmission. Background Technology

[0002] In risk management, banks face multiple challenges, such as processing massive amounts of data and real-time risk monitoring. Specifically, they typically need to conduct credit risk assessments on companies to measure the investment risk. In addition, banks' business systems usually need to monitor companies' account transaction behavior in real time to promptly detect abnormal transactions and issue early warnings.

[0003] Current technologies typically employ centralized computing architectures, which suffer from slow processing speeds when dealing with large-scale data. Assessment tasks that would normally take several days to complete severely restrict banks' decision-making efficiency. Furthermore, centralized architectures struggle to fully utilize the distributed computing resources of cloud computing data centers, resulting in low resource utilization and increased operating costs for banks. In addition, when faced with a large number of unusual transactions involving large sums, frequent transfers to diverse recipients within a short period, the system may fail to respond promptly, leading to delayed alerts and an inability to effectively prevent further fund losses, potentially causing economic damage to both the bank and its customers.

[0004] Therefore, there is an urgent need for a method that can improve data processing efficiency and optimize computing power allocation in order to enhance the level of bank risk management. Summary of the Invention

[0005] This application provides a computing power optimization method and system for cloud computing data center transmission, which can solve one of the above-mentioned problems in the prior art.

[0006] In a first aspect, embodiments of this application provide a method for optimizing computing power in cloud computing data center transmission, including: A three-level caching mechanism is adopted to cache corporate bank data and preprocess the corporate bank data. A deep scheduling model is constructed based on computing resource indicators, transmission path indicators, and task dynamic indicators. Based on business needs and combined with preprocessed corporate bank data, the deep scheduling model dynamically adjusts the allocation of computing power and task scheduling in the cloud computing data.

[0007] Furthermore, the three-level caching mechanism includes SSD caching, memory queue caching, and partition caching; The three-level caching mechanism adopted to cache corporate bank data includes: Obtain corporate bank data and cache the corporate bank data sequentially to the SSD cache, the memory queue cache, and the partition cache; The data in the memory queue cache is prioritized according to its importance and urgency.

[0008] Furthermore, the preprocessing of the corporate bank data includes: During the data acquisition phase, the corporate bank data is normalized, and a matching engine is built using Verilog. The matching engine is then used to identify key data in the corporate bank data. Based on the aforementioned key data, the data type of the corporate bank data is determined; Based on the data type, different data sharding strategies are used to shard the corporate bank data.

[0009] Furthermore, the step of segmenting the corporate bank data according to the data type using different data segmentation strategies includes: For structured data, a columnar partitioning method is used; For time series data, a slicing method with overlapping time windows is used; For text-type data, semantic segmentation is used.

[0010] Furthermore, the construction of a deep scheduling model based on computing resource metrics, transmission path metrics, and task dynamic metrics includes: Based on the resource status of cloud computing data centers, multidimensional heterogeneous data indicators are constructed, including computing resource indicators, transmission path indicators, and task dynamic indicators. Based on the status and business needs of cloud computing data centers, an action space set is constructed, which includes multiple types of actions for computing power allocation and task scheduling. Based on the multidimensional heterogeneous data indicators and the action space set, the deep scheduling model is trained by combining a multi-objective optimization reward function.

[0011] Furthermore, the multi-objective optimization reward function is: ; in, This indicates the reward items for completing the task within the specified time. Indicates energy consumption incentives; This indicates a reward for achieving the service level agreement target rate. Indicates penalty items for changes in actions; This represents the CPU utilization balance term; , as well as This represents the weighting coefficient, used to measure the importance of different objectives.

[0012] Furthermore, the task completion time reward item is as follows: ; in, This indicates the task completion time specified in the service level agreement. Indicates the actual task completion time; The energy consumption incentive item is used to measure the energy consumption cost of a cloud computing data center during task execution, specifically: in, , as well as These represent the energy consumption of the CPU, GPU, and network during task execution, respectively. Indicates the unit price of electricity; The service level agreement compliance rate award is used to measure the on-time completion of tasks, specifically: Where I(·) represents an indicator function, which is 1 when the task is completed on time and 0 otherwise. This indicates the importance weight of each task.

[0013] Furthermore, based on business needs and combined with preprocessed corporate bank data, the deep scheduling model dynamically adjusts the computing power allocation and task scheduling in the cloud computing data, including: Based on historical load data, an ARIMA model is constructed. The ARIMA model is used to predict load data at a preset time. Combined with the elastic scaling algorithm, the first triggering condition of the scheduling action is determined through the deep scheduling model. Based on data characteristics and transmission requirements, a compression algorithm is dynamically selected to compress the corporate bank data. By monitoring transmission path metrics and combining them with a deep scheduling model, the second triggering condition for scheduling actions can be determined. Based on market information, a task scheduling priority list is generated to determine the computing power allocation scheme.

[0014] Furthermore, the step of generating a task scheduling priority list based on market information and determining a computing power allocation scheme includes: By monitoring financial market data streams in real time, market information is obtained, and the market information is preprocessed to generate market fluctuation data. Based on the market fluctuations, a preset threshold judgment method is used to determine a high volatility indicator signal; Based on the high-fluctuation identifier signal, a task scheduling priority list for business needs is determined, and a computing power allocation scheme is determined through the deep scheduling model.

[0015] Secondly, embodiments of this application provide a computing power optimization system for cloud computing data center transmission, comprising: First processing module: used to cache corporate bank data using a three-level caching mechanism and to preprocess the corporate bank data; The second processing module is used to build a deep scheduling model based on computing resource indicators, transmission path indicators, and task dynamic indicators. The third processing module is used to dynamically adjust the computing power allocation and task scheduling in the cloud computing data based on business needs and the preprocessed corporate bank data, through the deep scheduling model.

[0016] The beneficial effects of the embodiments of this application compared with the prior art are: This application discloses a computing power optimization method for cloud computing data center transmission. It employs a three-level caching mechanism to cache and transmit corporate bank data, and divides and distributes the corporate bank data to multiple computing nodes for parallel processing, thereby improving the data transmission and processing efficiency of the cloud computing data center. In addition, a deep scheduling model is constructed through state space and action space for computing power allocation and task scheduling in the cloud computing data center, thereby making full use of the computing power resources of the cloud computing data center, avoiding resource waste, improving resource utilization, and reducing the bank's operating costs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a computing power optimization method for cloud computing data center transmission according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a computing power optimization system for cloud computing data center transmission provided in an embodiment of the present invention. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] Please see Figure 1 As shown, this invention is a computing power optimization method for cloud computing data center transmission, comprising the following steps: S100. A three-level caching mechanism is adopted to cache corporate bank data and preprocess the corporate bank data. In this application, a three-level caching mechanism is used to cache and transmit corporate bank data, and the corporate bank data is divided and distributed to multiple computing nodes for parallel processing, thereby improving the data transmission and processing efficiency of the cloud computing data center.

[0026] In some embodiments, the three-level caching mechanism includes SSD caching, memory queue caching, and partition caching; The three-level caching mechanism for caching corporate bank data includes: Obtain corporate bank data and cache the corporate bank data sequentially to the SSD cache, the memory queue cache, and the partition cache; The data in the memory queue cache is prioritized according to its importance and urgency.

[0027] In this embodiment, corporate banking data for business needs such as corporate credit risk assessment and real-time transaction monitoring are collected from multiple data sources. For example, in corporate credit risk assessment, data sources typically include the bank's internal corporate transaction system, financial system, and external industry databases, government open data platforms, etc. For real-time transaction monitoring, the data source is mainly the bank's core transaction system, while corporate banking data specifically includes transaction data, financial statements, industry data, market data, regulatory policy documents, etc.

[0028] In this embodiment, the acquired corporate bank data is cached using a three-level caching mechanism. The SSD cache is used for data buffering during bursts of traffic, the memory queue cache is used for prioritizing the corporate bank data, and the partition cache is used for persistent storage. In a preferred embodiment, the SSD cache uses high-performance NVMe SSDs with read / write speeds of several GB / s, which can effectively handle data surges in a short period of time. The memory queue cache adopts a lock-free queue design to reduce inter-thread contention and improve data throughput. For the partition cache, multiple partitions are deployed using a Kafka cluster. The number of partitions is configured according to the amount of corporate bank data and load balancing requirements to ensure the reliability and scalability of persistent data storage.

[0029] Specifically, the acquired corporate banking data first enters the SSD cache, utilizing the high-speed read and write capabilities of the SSD to quickly receive the data. Subsequently, the data enters the memory queue and is sorted using a priority sorting algorithm to ensure that high-priority data is processed first. Finally, the data is stored in a Kafka partition. Kafka uses a multi-replica mechanism to ensure data reliability, and the number of replicas can be adjusted according to business needs.

[0030] More specifically, for data in the memory queue cache, priorities are set according to data importance and urgency. Specifically, the data in the memory queue is classified and a basic priority is set according to business importance. For example, real-time data such as real-time trading data, market depth data, and real-time market snapshots are set as high priority; data that is updated periodically, such as financial statement data, intraday trading data, and macroeconomic indicators, are set as medium priority; and data with strong stability, such as historical daily candlestick charts, static reference data, and non-real-time announcements, are set as low priority.

[0031] In some embodiments, the preprocessing of the corporate bank data includes: During the data acquisition phase, the corporate bank data is normalized, and a matching engine is built using Verilog. The matching engine is then used to identify key data in the corporate bank data. Based on the aforementioned key data, the data type of the corporate bank data is determined; Based on the data type, different data sharding strategies are used to shard the corporate bank data.

[0032] In this embodiment, the acquired corporate bank data needs to be preprocessed before caching to standardize various data types and improve the ability to call and process data subsequently. Specifically, corporate bank data includes report-type data and text-type data. Each type of corporate bank data undergoes normalization. In report-type data, for numerical data such as transaction amounts, a linear normalization method is used to map the transaction amount to the [0, 1] range. For text-type data, a preset encoding rule is used. For example, in one possible embodiment, a high-precision geocoding library is used to geocode IP addresses, allowing for rapid acquisition of geographical location information based on the IP address. For text-type data, such as regulatory policy documents or market analysis reports, the numerical data is normalized using the same normalization method as in report-type data to ensure data consistency. The text-type data is processed using methods such as encoding standardization, case sensitivity handling, number / symbol standardization, and stop word filtering, ultimately outputting normalized text.

[0033] In this embodiment, for normalized transaction data, financial statements, industry data, market data, and other report-type data, a matching engine is used for data mapping to facilitate subsequent data matching, searching, and storage. Specifically, a regular expression compiler, such as Flex or a custom tool, is used to convert regular expressions into Verilog-recognizable FSMs. For example, in one embodiment, a transaction ID is compiled as / TXN\d+ / , which is detected by the state machine as T→X→N followed by a number. Similarly, an account number is compiled as / ACC-\d+ / , which is detected as A→C→C→- followed by a number. Likewise, for numeric data within text-type data, a matching engine is used for data mapping.

[0034] In some embodiments, the step of segmenting the corporate bank data according to the data type using different data segmentation strategies includes: For structured data, a columnar partitioning method is used; For time series data, a slicing method with overlapping time windows is used; For text-type data, semantic segmentation is used.

[0035] In this embodiment, different data sharding strategies are adopted according to the data type to shard the data, and then store it in the partition cache. Specifically, the report type data includes structured data and time series data. It can be understood that structured data is a data type that is not related to time series, such as financial statements, while time series data is a data type that is related to time series, such as real-time transaction data.

[0036] In this embodiment, when mapping data through a matching engine, the data is classified into structured data, time-series data, and text-type data. Specifically, the data structure characteristics are analyzed. If the data contains a fixed field structure, it is marked as structured data; if the data contains time series data, it is marked as time-series data; and if the data contains natural language text, it is marked as text-type data. This yields the data type classification results. Based on the data type classification results, the corresponding sharding strategy is matched. Specifically, structured data corresponds to columnar sharding, time-series data corresponds to sharding with overlapping time windows, and text-type data is assigned semantic sharding, thereby determining the processing strategy for each data type.

[0037] In this embodiment, a vertical partitioning operation is used for structured data, decomposing the data table into multiple sub-tables according to the column attribute dimension. Each sub-table contains related column attribute combinations, thus obtaining a set of fragments of structured data.

[0038] In this embodiment, an overlapping time window segmentation operation is used for time series data. By presetting the time window length and overlap ratio parameters, multiple overlapping time periods are generated in the order of the time axis, and the time series data is allocated to the corresponding time window to obtain a set of time series data fragments.

[0039] In this embodiment, semantic similarity grouping is used for text data. A word vector model is used to calculate the semantic similarity matrix between texts. Based on the similarity threshold, texts with similar semantics are classified into the same group to obtain a fragmented set of text data.

[0040] In some embodiments, a grouping rule validator is used to check the sharding results of each type of data. If the number of shards exceeds the preset range, the sharding parameters are adjusted and the sharding operation is re-executed. If the sharding content is duplicated, deduplication is performed to determine the final data sharding scheme.

[0041] In this embodiment, the data of each enterprise bank is physically divided according to the data sharding scheme to generate independent data shard files, and a sharding index mapping relationship is established to obtain a complete sharded data storage structure. Furthermore, during data processing, each piece of data is distributed to different computing nodes for processing according to the data sharding scheme. For example, for structured data, such as financial statements, core fields such as the debt-to-asset ratio are distributed to multiple financial dedicated computing nodes according to the sharding results, or transaction flow data is sharded through time windows and distributed to multiple stream processing optimization nodes for distributed processing to improve the efficiency of data processing.

[0042] S200: Based on computing resource indicators, transmission path indicators, and task dynamic indicators, a deep scheduling model is constructed. In this application, a deep scheduling model is constructed through state space and action space for computing power allocation and task scheduling in cloud computing data centers. This enables full utilization of computing power resources in cloud computing data centers, avoids resource waste, improves resource utilization, and reduces the operating costs of banks.

[0043] In some embodiments, step S200 above includes: Based on the resource status of cloud computing data centers, multidimensional heterogeneous data indicators are constructed, including computing resource indicators, transmission path indicators, and task dynamic indicators. Based on the status and business needs of cloud computing data centers, an action space set is constructed, which includes multiple types of actions for computing power allocation and task scheduling. Based on the multidimensional heterogeneous data indicators and the action space set, the deep scheduling model is trained by combining a multi-objective optimization reward function.

[0044] In this embodiment, when constructing the deep scheduling model, a state space and an action space are defined, and then the multidimensional heterogeneous data indicators in the state space are mapped to scheduling actions in the action space to achieve joint optimization of computation and transmission.

[0045] In this embodiment, based on the resource status of the cloud computing data center, multi-dimensional heterogeneous data indicators are established, specifically including computing resource indicators, transmission path indicators, and task dynamic indicators. The computing resource indicators are used to reflect the computing load of each node in real time, providing a basis for task migration and node expansion. In a preferred embodiment, these indicators specifically include CPU utilization, memory pressure index, and GPU utilization of each node. The transmission path indicators are used to monitor network status and support decisions such as path switching and bandwidth reservation. In a preferred embodiment, these indicators specifically include storage I / O throughput, transmission path indicators, core link bandwidth utilization, end-to-end latency, packet loss rate, and path congestion level. The task dynamic indicators are used to ensure that the scheduling strategy is linked to real-time load changes. In a preferred embodiment, these indicators specifically include the length of the pending task queue, the estimated task time, the task data volume, and the task priority.

[0046] In a preferred embodiment, the action space set specifically includes node expansion, node shrinkage, task migration, path switching, compression strategy adjustment, bandwidth reservation, cache preheating, frequency adjustment, and redundant transmission. For node expansion, new instances are launched by calling cloud platform APIs to increase computing resource supply. For node shrinkage, idle nodes are released to reduce energy consumption costs. For task migration, specified tasks are migrated to low-load nodes to ensure load balancing. For path switching, data transmission paths are dynamically adjusted based on network status to reduce transmission latency. For compression strategy adjustment, compression algorithms are dynamically selected to reduce the amount of data transmitted. For bandwidth reservation, end-to-end bandwidth is reserved for high-priority tasks to improve the compliance rate of service level agreements. For cache preheating, the data required for computation is preloaded into the target node's memory to reduce computation startup latency. For frequency adjustment, CPU / GPU frequencies are dynamically adjusted to reduce energy consumption costs. For redundant transmission, multi-path redundant transmission is enabled for critical data to improve transmission reliability. Understandably, different scheduling actions are related to ensure the reliability of business implementation. For example, task migration usually requires joint path switching to ensure that low-load nodes and low-latency paths are selected during migration, thereby improving the efficiency of data computation and data transmission.

[0047] In this embodiment, by training a deep scheduling model, the model can associate the resource status in cloud computing data with business needs, thereby allocating computing power and scheduling tasks. Specifically, a suitable training model is selected, such as a deep Q-network or a policy gradient (PG) model. Then, the model parameters are randomly initialized, and various preset computing power allocation and task scheduling strategies are run in a simulation environment. Data on state, action, reward, and next state are collected and stored as experience in a replay buffer. Training data is randomly sampled, and the network parameters are optimized through gradient descent to minimize the loss function. Through multiple iterations of training, the model converges, that is, the reward value of the reward function tends to stabilize.

[0048] In some embodiments, the multi-objective optimization reward function is measured by a task completion time reward, an energy consumption reward, a service level agreement compliance rate reward, an action change penalty, and a CPU utilization balancing item.

[0049] Specifically, the multi-objective optimization reward function is: ; in, This indicates the reward items for completing the task within the specified time. Indicates energy consumption incentive items; This indicates a reward for achieving the service level agreement target rate. Indicates penalty items for changes in actions; This represents the CPU utilization balance term; , as well as This represents the weighting coefficient, used to measure the importance of different objectives.

[0050] In this embodiment, a multi-objective optimization reward function is designed with the goals of minimizing latency, maximizing resource utilization, minimizing cost, and maximizing reliability. This allows the deep scheduling model to fully measure task completion time and resource utilization, thereby avoiding resource idleness or overload, reducing unnecessary expansion or redundant transmission, and ensuring successful transmission of critical tasks.

[0051] Specifically, Used to measure task completion time Measuring energy consumption costs Used to measure the reliability of task completion, further, through This is used to ensure the stability of various scheduling actions during the scheduling process of cloud computing data centers, avoiding frequent changes in actions, and through... Balance CPU utilization to avoid situations where some CPU cores are overloaded while others are idle, thereby improving resource utilization and reliability.

[0052] Specifically, For each action k, if the action at the current time step t is different from the action at the previous time step t−1, the count is incremented by 1, and this is combined with the penalty coefficient -0.01 to obtain the action change penalty term. It can be understood that the action k here corresponds to the scheduled action in the action space set.

[0053] Specifically, ,in, This represents the variance of CPU utilization. The larger the variance, the more unbalanced the CPU utilization. Based on this, combined with the penalty coefficient of -0.05, we obtain the CPU utilization balance term.

[0054] In some embodiments, the task completion time reward is measured by comparing the task completion time specified in the service level agreement with the actual task completion time; Specifically, the task completion time reward items are as follows: ; in, This indicates the task completion time specified in the service level agreement. Indicates the actual task completion time; Specifically, when the actual task completion time Less than or equal to the time specified in the Service Level Agreement At that time, the reward followed The increase decreases linearly, while when When the value is 0, the reward reaches its maximum value of 1. Furthermore, when... > At that time, the reward is negative, and as... Exceeding The degree of increase linearly decreases. It is a penalty coefficient used to control exceeding limits. The severity of punishment after a certain period of time.

[0055] In some embodiments, the energy consumption reward item is used to measure the energy consumption cost of a cloud computing data center during task execution, specifically: Where i represents the i-th computing node. , as well as These represent the energy consumption of the CPU, GPU, and network of computing node i during task execution, respectively. Indicates the unit price of electricity; Specifically, since lower energy consumption indicates lower energy costs, the energy consumption reward is set to a negative value. For each computing node i, the energy consumption is calculated separately for the CPU, GPU, and network, then the energy consumption of all components is summed and compared with the unit electricity price. Multiplying these values ​​yields the total energy consumption cost. Further, for CPU and GPU components, the product of the corresponding power P and usage time t is calculated to obtain the corresponding energy consumption. For network components, the network's energy consumption is obtained by calculating the product of power and transmitted data volume. Here, power... It is calculated based on the amount of data transmitted, bandwidth, and power of each port. Specifically, Where Data represents the amount of data to be transmitted. Indicates network bandwidth, via Obtain the data transmission time, and then combine it with the power consumption of a single network port. The energy consumption of a single network port during data transmission.

[0056] The service level agreement compliance rate award is used to measure the on-time completion of tasks, specifically: Where I(·) represents an indicator function, which is 1 when the task is completed on time and 0 otherwise, and j represents the j-th task. This represents the importance weight of the j-th task.

[0057] Specifically, the Service Level Agreement (SLA) sets key performance indicators (SPIs) for cloud computing data centers to improve service quality, such as availability, performance, and response time. The SLA ultimately determines whether the allocation of computing power and task scheduling meet business needs. A reward is awarded based on the percentage of tasks meeting the SLA's time requirements out of the total number of tasks, combined with task priority weights.

[0058] S300. Based on business needs and combined with preprocessed corporate bank data, the deep scheduling model is used to dynamically adjust the computing power allocation and task scheduling in the cloud computing data.

[0059] In this embodiment, the bank's business needs are mapped to task dynamic indicators in the deep scheduling model. Specifically, a preset priority label is assigned to each of the bank's business needs, corresponding to the task priority in the task dynamic indicators. In a preferred embodiment, each priority label is allocated corresponding computing resources to ensure that each priority level can obtain sufficient computing power resources for processing. For example, in a possible embodiment, the priority label for real-time account freezing is level 4. The cloud computing data center allocates a dedicated GPU node + 100G RDMA network computing power indicator to it based on preset allocation rules. During task execution, the transaction behavior of the enterprise account is monitored in real time. When an abnormal transaction pattern is detected, a resource preemption task is immediately triggered, and the specified resources are quickly allocated through the cloud computing platform API to ensure that the task starts within a millisecond response time.

[0060] In some embodiments, step S300 above includes: Based on historical load data, an ARIMA model is constructed. The ARIMA model is used to predict load data at a preset time. Combined with the elastic scaling algorithm, the first triggering condition of the scheduling action is determined through the deep scheduling model. Based on data characteristics and transmission requirements, a compression algorithm is dynamically selected to compress the corporate bank data. By monitoring transmission path metrics and combining them with a deep scheduling model, the second triggering condition for scheduling actions can be determined. Based on market information, a task scheduling priority list is generated to determine the computing power allocation scheme.

[0061] In this embodiment, when the cloud computing data center processes the corresponding tasks according to business needs, it needs to monitor the computing resource indicators of the cloud computing data center in real time and determine whether to perform scheduling actions such as node expansion, node shrinkage or task migration by calling the deep scheduling model, which is the first triggering situation.

[0062] In this embodiment, by constructing an ARIMA model and combining dynamic load balancing and elastic scaling algorithms, the computing resources of the cloud computing data center are fully utilized, avoiding resource waste, improving resource utilization, and reducing the bank's operating costs. Therefore, by observing the dynamic changes in the cloud computing data center load over time, it is determined whether the number of nodes needs to be adjusted or tasks need to be migrated to balance performance and cost. Historical load data is obtained, specifically including the number of task requests per minute and the CPU utilization, memory pressure index, and GPU utilization of each computing node at the corresponding time point. The historical load data is filled using linear interpolation, and the ADF test is used to determine whether the sequence is stationary, thereby achieving preprocessing of the historical load data.

[0063] More specifically, the ARIMA model is an autoregressive integral moving average model, which includes an autoregressive model (AR), a differencing process (I), and a moving average model (MA). When constructing the ARIMA model, the (p, d, q) parameters are determined through the ACF and PACF plots, where p represents the autoregressive order, specifically the influence of the load values ​​at the past p time points on the current value, d represents the differencing order, the number of differencing steps used to make the sequence stable, and q represents the moving average order, specifically the influence of the error terms at the past q time points on the current value. Then, the ARIMA model is fitted using the preprocessed historical load data to minimize the mean squared error, which is used to predict the load data within a preset time period.

[0064] In this embodiment, an elastic scaling algorithm is used to calculate a resource scaling factor to determine whether node expansion, node shrinking, or task migration is necessary. Specifically, the calculation formula for the elastic scaling algorithm is as follows: Where Load(t) represents the current or predicted load data, and Capacity represents the total capacity of the current node, used to measure the maximum task processing capacity of the current cloud computing data center. This represents the load change rate, specifically calculated using the load differential predicted by ARIMA. a and b represent adjustment weights, which can be adjusted according to business needs.

[0065] Furthermore, if the resource scaling factor is less than a preset threshold This indicates that there are idle computing resources in the current cloud computing data center. In this case, node scaling down is performed using a deep scheduling model. If the resource scaling factor is greater than a preset threshold... This indicates that the current cloud computing data center is computationally intensive. At this time, by calling the deep scheduling model, it is determined how the cloud computing data center should trigger scheduling actions to minimize system latency, maximize resource utilization, minimize cost, and maximize reliability. Specifically, the cloud computing data center may directly trigger node expansion actions, or it may maximize resource utilization through task migration between nodes. It is understandable that when one scheduling action is triggered, other scheduling actions are usually linked to maximize the computing power allocation of the cloud computing data center.

[0066] In this embodiment, corporate bank data is compressed to ensure data transmission efficiency. Furthermore, different compression algorithms are employed to address different data characteristics and transmission requirements, minimizing the impact of compression on data integrity while maintaining data transmission efficiency. Specifically, for structured data, Delta+RLE encoding is used. Delta encoding is suitable for data with small changes in field values ​​and low dispersion, compressing data by calculating the difference between adjacent field values. RLE encodes consecutively repeated Delta values, recording the repetition count and value. Additionally, for mixed-type fields combining numerical and string data in structured data, ZSTD Level 12 is used to uniformly encode both numerical and string fields. This employs dictionary compression and entropy encoding techniques, supporting data block compression without waiting for all data to load, reducing memory usage, and making it suitable for large-scale data. Furthermore, for time series data compression, Gorilla encoding is used. Specifically, Delta encoding is used to compress timestamps, recording the difference from the previous timestamp. Highly continuous timestamps (such as milliseconds) are compressed using binary encoding. Then, XOR encoding is used to compress numerical values, recording the difference from the previous value. Repeated numerical values ​​are encoded using RLE encoding. Furthermore, for text data, a predefined dictionary is used to compress repetitive text. The dictionary can be dynamically updated to adapt to different texts. Huffman encoding or arithmetic encoding is used to compress the text. Combining dictionary compression improves the compression ratio.

[0067] In some embodiments, the corporate bank data is encrypted and its key is stored before data compression to ensure data security. For example, the BLAKE3 hash value is calculated before and after compression, and consistency is verified through a smart contract. Simultaneously, encryption and data compression work together to ensure lossless compression and guarantee data integrity and reliability. Specifically, before data compression, the BLAKE3 hash value of the original data is calculated and stored; after compression, the hash value of the compressed data is calculated and compared with the original hash value. The smart contract is deployed on the blockchain to achieve trusted verification of the hash value, thereby ensuring the confidentiality, integrity, and availability of data during transmission and effectively preventing security risks such as data leakage and tampering.

[0068] In this embodiment, when executing relevant business requirements, the deep scheduling model monitors the transmission path indicators to determine whether a path switching action decision is needed, i.e., the second triggering situation. In some possible embodiments, if one or more of the transmission path indicators, such as storage IO throughput, transmission path indicators, core link bandwidth utilization, end-to-end latency, packet loss rate, and path congestion level, are higher than a preset threshold, the deep scheduling model invokes the path switching scheduling action.

[0069] Furthermore, when path switching is required, a fusion of segmented routing and ant colony optimization algorithms is employed to dynamically avoid high-load links and select the optimal data transmission path. SRv6 provides flexible network programming capabilities, while the ant colony optimization algorithm dynamically searches for the optimal path by simulating ant foraging behavior. Specifically, an initial path is constructed based on network topology and link state information. Then, through iterative ant search, path quality is evaluated based on metrics such as link bandwidth, latency, and packet loss rate, gradually optimizing path selection, avoiding high-load links, and ultimately determining the optimal data transmission path. When the deep scheduling model triggers a path switching action, the flow table entries are updated via the OpenFlow protocol according to the data transmission path, thereby achieving dynamic adjustment of the data transmission path.

[0070] In some embodiments, the step of generating a task scheduling priority list based on market information and determining a computing power allocation scheme includes: By monitoring financial market data streams in real time, market information is obtained, and the market information is preprocessed to generate market fluctuation data. Based on the market fluctuations, a preset threshold judgment method is used to determine a high volatility indicator signal; Based on the high-fluctuation identifier signal, a task scheduling priority list for business needs is determined, and a computing power allocation scheme is determined through the deep scheduling model.

[0071] In this embodiment, market information is further introduced into the task dynamic indicators of the cloud computing data center, and the priority of each task in the task dynamic indicators is dynamically adjusted, so that the bank can quickly identify potential risks, issue early warning information, and prevent the loss of funds.

[0072] In this embodiment, the market information specifically refers to the market information of the industry corresponding to the enterprise being monitored by the bank. Specifically, the transaction data of the relevant industry in the securities trading system is collected in real time through the Kafka stream processing platform, including timestamp, price, and trading volume fields, to generate a volatility dataset. This dataset is written to a Redis in-memory database to form a raw data cache. Outliers in the raw data are removed using the interquartile range method, and the price series is processed by Z-score standardization to output a structured volatility dataset. Using a 30-minute sliding window, the standard deviation of the price within each window is calculated as the volatility feature value, generating a time-varying data set. The feature sequence of the stamp is compared point by point with a preset volatility threshold. Points exceeding the threshold are marked as candidate high volatility points. Three or more consecutive candidate high volatility points are aggregated. If the coverage time exceeds 2 hours, it is determined to be a high volatility period. A high volatility identification signal is output, and the start and end times of the high volatility period are marked in the volatility dataset to form a labeled volatility record. This record is input into a pre-trained linear kernel support vector machine model, which outputs three prediction labels: rise / fall / oscillation. The prediction labels are compared with the historical results of similar volatility events in the past 30 days. When the matching degree is higher than 60%, it is marked as a high-risk level.

[0073] Furthermore, based on the volatility dataset corresponding to the high volatility indicator signals marked as high-risk levels, the task priority of each task in the task dynamic indicators is determined, a scheduling task priority list is generated, and the computing power allocation scheme that needs to be adjusted is determined. Specifically, the volatility dataset corresponding to the high volatility indicator signals marked as high-risk levels is matched with the task dynamic indicators of the deep scheduling model, and computing power resources are reallocated to market-sensitive businesses, such as high-frequency trading and risk assessment. Then, a load balancing algorithm is used to distribute tasks to computing nodes and generate a computing power priority allocation scheme.

[0074] In some embodiments, the price standard deviation of the volatility dataset corresponding to the high volatility identifier signal marked as high risk level is calculated as the volatility feature value. When the volatility feature value is higher than the high volatility signal of high risk level in the deep scheduling model, the deep scheduling model reallocates computing power, that is, increases the priority of market-sensitive business and allocates corresponding resources according to the risk level of the task, ensuring that high priority tasks can obtain sufficient computing power resources, thereby improving the data processing efficiency and business response capability of corporate banks, and thus effectively managing risks and improving the stability and reliability of financial transactions.

[0075] Please see Figure 2 As shown, the present invention also provides a computing power optimization system for cloud computing data center transmission, the system comprising: First processing module 203: used to cache corporate bank data using a three-level caching mechanism and preprocess the corporate bank data; The second processing module 202 is used to construct a deep scheduling model based on computing resource indicators, transmission path indicators, and task dynamic indicators. The third processing module 201 is used to dynamically adjust the computing power allocation and task scheduling in the cloud computing data based on business needs and the preprocessed corporate bank data, through the deep scheduling model.

[0076] It is understandable that, such as Figure 1 The content of the computing power optimization method embodiment for cloud computing data center transmission shown is applicable to the computing power optimization system embodiment for cloud computing data center transmission. The specific functions implemented by the computing power optimization system embodiment for cloud computing data center transmission are the same as those shown in the figure. Figure 1 The embodiment of the computing power optimization method for cloud computing data center transmission shown is the same, and the beneficial effects achieved are the same as those described above. Figure 1 The beneficial effects achieved by the embodiment of the computing power optimization method for cloud computing data center transmission shown are also the same.

[0077] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0078] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0079] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for optimizing computing power in cloud computing data center transmission, characterized in that, include: A three-level caching mechanism is adopted to cache corporate bank data and preprocess the corporate bank data. A deep scheduling model is constructed based on computing resource indicators, transmission path indicators, and task dynamic indicators. According to business needs and combined with preprocessed corporate bank data, the computing power allocation and task scheduling of the cloud computing data center are dynamically adjusted through the deep scheduling model. The construction of a deep scheduling model based on computing resource indicators, transmission path indicators, and task dynamic indicators includes: constructing multi-dimensional heterogeneous data indicators based on the resource status of the cloud computing data center; constructing an action space set based on the status and business requirements of the cloud computing data center; training the deep scheduling model based on the multi-dimensional heterogeneous data indicators and the action space set, combined with a multi-objective optimization reward function; the multi-objective optimization reward function measures the performance through task completion time reward, energy consumption reward, service level agreement compliance rate reward, action change penalty, and CPU utilization balancing item; the task completion time reward is measured by comparing the task completion time specified in the service level agreement with the actual task completion time; the action change penalty is used to ensure the stability of each scheduling action during the scheduling process of the cloud computing data center; the CPU utilization balancing item is used to balance CPU utilization. The process of dynamically adjusting the computing power allocation and task scheduling of the cloud computing data center based on business needs and preprocessed corporate bank data through the deep scheduling model includes: constructing an ARIMA model based on historical load data, which is used to predict load data at a preset time; determining the first triggering condition for scheduling actions through the deep scheduling model in conjunction with an elastic scaling algorithm; dynamically selecting a compression algorithm to compress the corporate bank data based on data characteristics and transmission requirements; determining the second triggering condition for scheduling actions by monitoring transmission path indicators and combining the deep scheduling model; and generating a task scheduling priority list based on market information to determine the computing power allocation scheme.

2. The method as described in claim 1, characterized in that, The three-level caching mechanism includes SSD caching, memory queue caching, and partition caching; The three-level caching mechanism adopted to cache corporate bank data includes: Obtain corporate bank data and cache the corporate bank data sequentially to the SSD cache, the memory queue cache, and the partition cache; The data in the memory queue cache is prioritized according to its importance and urgency.

3. The method as described in claim 1, characterized in that, The preprocessing of the corporate bank data includes: During the data acquisition phase, the corporate bank data is normalized, and a matching engine is built using Verilog. The matching engine is then used to identify key data in the corporate bank data. Based on the aforementioned key data, the data type of the corporate bank data is determined; Based on the data type, different data sharding strategies are used to shard the corporate bank data.

4. The method as described in claim 3, characterized in that, The step of segmenting the corporate bank data according to the data type using different data segmentation strategies includes: For structured data, a columnar partitioning method is used; For time series data, a slicing method with overlapping time windows is used; For text-type data, semantic segmentation is used.

5. The method as described in claim 1, characterized in that, The task completion time reward is measured by comparing the task completion time specified in the service level agreement with the actual task completion time. The energy consumption incentive is used to measure the energy consumption cost of a cloud computing data center during task execution; The service level agreement compliance rate reward item is used as an indicator to measure the timely completion of tasks.

6. The method as described in claim 5, characterized in that, The process of generating a task scheduling priority list based on market information and determining a computing power allocation scheme includes: By monitoring financial market data streams in real time, market information is obtained, and the market information is preprocessed to generate market fluctuation data. Based on the market fluctuations, a preset threshold judgment method is used to determine a high volatility indicator signal; Based on the high-fluctuation identifier signal, a task scheduling priority list for business needs is determined, and a computing power allocation scheme is determined through the deep scheduling model.

7. A computing power optimization system for cloud computing data center transmission, characterized in that, include: First processing module: used to cache corporate bank data using a three-level caching mechanism and to preprocess the corporate bank data; The second processing module is used to build a deep scheduling model based on computing resource indicators, transmission path indicators, and task dynamic indicators. The third processing module is used to dynamically adjust the computing power allocation and task scheduling of the cloud computing data center based on business needs and the preprocessed corporate bank data, through the deep scheduling model. The construction of a deep scheduling model based on computing resource indicators, transmission path indicators, and task dynamic indicators includes: constructing multi-dimensional heterogeneous data indicators based on the resource status of the cloud computing data center; constructing an action space set based on the status and business requirements of the cloud computing data center; training the deep scheduling model based on the multi-dimensional heterogeneous data indicators and the action space set, combined with a multi-objective optimization reward function; the multi-objective optimization reward function measures the performance through task completion time reward, energy consumption reward, service level agreement compliance rate reward, action change penalty, and CPU utilization balancing item; the task completion time reward is measured by comparing the task completion time specified in the service level agreement with the actual task completion time; the action change penalty is used to ensure the stability of each scheduling action during the scheduling process of the cloud computing data center; the CPU utilization balancing item is used to balance CPU utilization. The process of dynamically adjusting the computing power allocation and task scheduling of the cloud computing data center based on business needs and preprocessed corporate bank data through the deep scheduling model includes: constructing an ARIMA model based on historical load data, which is used to predict load data at a preset time; determining the first triggering condition for scheduling actions through the deep scheduling model in conjunction with an elastic scaling algorithm; dynamically selecting a compression algorithm to compress the corporate bank data based on data characteristics and transmission requirements; determining the second triggering condition for scheduling actions by monitoring transmission path indicators and combining the deep scheduling model; and generating a task scheduling priority list based on market information to determine the computing power allocation scheme.

Images