Collaborative method and system for containerized mixed deployment of database and financial software

Abstract

The invention relates to the technical field of business and financial fusion, in particular to a collaboration method and system for containerized mixed deployment of a database and financial software. The method comprises the following steps: respectively packaging a database and financial software into independent containers and carrying out mixed deployment on a container arrangement platform; establishing a cooperative communication mechanism between the two to realize data synchronization and resource dynamic cooperative distribution; a deep learning model is integrated to analyze business data and system load in real time, and container resource allocation and data access strategies are dynamically optimized. According to the method, deep coupling of business and financial applications and the underlying database in a deployment layer is realized through a containerization technology, intelligent resource management and optimization are realized by utilizing deep learning, the problems of data islands and performance bottlenecks are effectively solved, the requirements of'business and financial fusion 'on real-time performance, consistency and high efficiency are met, and the method is suitable for popularization and application. And the resource utilization rate, the automatic operation and maintenance level and the overall performance of the system are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industry and finance integration, and particularly relates to a database and financial software containerized mixed deployment cooperation method and system. BACKGROUND

[0002] With the deepening of enterprise digital transformation, industry and finance integration has become the core demand of modern enterprise management, which requires real-time synchronization of data, seamless connection of processes and collaborative management of resources between business systems and financial systems. However, in the traditional enterprise IT architecture, business databases and financial software usually adopt an independent deployment mode, that is, business systems (such as ERP, CRM) and financial systems (such as accounting and fund management) are deployed on different physical servers or virtual machines, and batch data interaction is performed through pre-defined interfaces. This loosely coupled architecture has many inherent defects in practice. First, it is easy to form a "data island", and transaction data generated on the business side cannot be transmitted to the financial side in real time, resulting in significant lag in financial accounting, report generation and other processes, making it difficult to support real-time decision-making by managers. Secondly, the batch interface data transmission method will bring huge performance bottlenecks during the business peak period, not only the data consistency is difficult to guarantee, but also the system response is slow, which affects the core operation efficiency of the enterprise. To solve these problems, existing technologies try to use some preliminary integration solutions, such as application integration based on enterprise service bus or data extraction using ETL tools, but these solutions mostly stay at the application layer or data layer and cannot fundamentally solve the isolation problem at the deployment level. In recent years, the development of containerization technologies such as Docker and container orchestration platforms such as Kubernetes has brought agility and flexibility to application deployment, and some solutions have tried to containerize the database or financial software separately. However, existing containerization solutions usually only simply package the application and do not realize the deep cooperation of the database and financial software in the containerized environment. They still regard the two as independent and scalable services, ignoring their special requirements for data strong consistency, low latency communication and resource linkage deployment in the "industry and finance integration" scenario. This containerized deployment often fails to effectively break down the data island and may even introduce new performance bottlenecks due to resource competition. Therefore, there is an urgent need in the field for a new technical solution that can achieve deep coupling and resource cooperation between industry and finance applications and underlying databases at the deployment level, thereby directly responding to the stringent requirements of real-time, consistency and efficiency in "industry and finance integration".

[0003] Therefore, the prior art still needs further development. SUMMARY

[0004] The present application aims to overcome the above technical deficiencies and provide a database and financial software containerized mixed deployment cooperation method and system to solve the problems existing in the prior art.

[0005] To achieve the above technical purposes, according to a first aspect of the present application, a database and financial software containerized mixed deployment cooperation method is provided, comprising: S1, the database and financial software are respectively packaged into independent containers, and mixed deployment is carried out on the container orchestration platform to form a database container and a financial software container; S2, a cooperative communication mechanism is established between the database container and the financial software container to perform data synchronization and dynamic cooperative allocation of resources; S3, based on a deep learning model, real-time analysis of business data and system load is performed to dynamically optimize container resource allocation and data access strategy.

[0006] Specifically, the container orchestration platform in step S1 is implemented based on Kubernetes, and the database container and the financial software container are deployed in groups through the Pod mechanism of Kubernetes to ensure local optimization.

[0007] Specifically, the cooperative communication mechanism in step S2 adopts service mesh technology to establish a bidirectional authentication gRPC channel between the database container and the financial software container to realize low-latency data exchange.

[0008] Specifically, the deep learning model in step S3 is a long short-term memory network (LSTM) for predicting resource demand during business peak and triggering automatic container scaling operation in advance.

[0009] Specifically, the dynamic cooperative allocation of resources in step S2 includes dynamically adjusting the CPU and memory resource quotas of the database container and the financial software container based on real-time monitoring indicators to achieve load balancing.

[0010] Specifically, the real-time monitoring indicators include container-level CPU usage, memory usage, network I / O and disk I / O, which are collected and stored through the Prometheus system.

[0011] Specifically, it also includes triggering an early warning mechanism when the real-time monitoring indicators exceed the preset threshold, and performing anomaly detection and self-healing operation through the deep learning model to automatically reschedule container instances.

[0012] Specifically, the data synchronization in step S2 adopts a distributed transaction protocol to ensure strong consistency of financial data transmission between containers.

[0013] Specifically, the distributed transaction protocol is implemented based on two-phase commit (2PC), and the transaction commit strategy is optimized through the deep learning model to reduce lock contention and delay.

[0014] According to a second aspect of the present application, a collaborative system for containerized mixed deployment of databases and financial software is provided, comprising: a containerized deployment module for packaging databases and financial software into independent containers and performing mixed deployment on a container orchestration platform; a collaborative communication module for establishing data synchronization and resource dynamic collaborative allocation mechanisms between database containers and financial software containers; a deep learning optimization module integrating a deep learning model for real-time analysis of business data and system load, dynamic adjustment of container resource allocation and data access strategy.

[0015] Advantages: The collaborative method and system for containerized mixed deployment of databases and financial software provided by the present application have brought significant and multi-faceted advantages compared to the prior art. First, the present application fundamentally solves the two core problems of "data island" and "performance bottleneck" through containerized mixed deployment and collaborative communication mechanisms. By using the Kubernetes Pod mechanism to tightly co-locate databases and associated financial software containers, and combining service mesh and gRPC two-way authentication channels, efficient and secure communication between containers is achieved, enabling business data to be synchronized to the financial end with extremely low latency, ensuring real-time and consistency of data, and providing underlying architecture support for true "industry and finance integration". Second, the present application realizes intelligent and forward-looking resource management by introducing a deep learning model. By using an LSTM network to predict business load trends and trigger automatic scaling of container resources, the system changes from passive response to proactive planning, effectively avoiding performance degradation during peak business periods and avoiding resource waste during non-peak periods, thereby significantly improving the resource utilization of the entire cluster and the stability and reliability of the system. Third, the present application greatly improves the automation level and robustness of system operation and maintenance through fine-grained monitoring and early warning and self-healing mechanisms. Based on real-time monitoring indicators and preset thresholds in multiple dimensions, combined with self-encoder for anomaly detection, the system can timely detect problems and automatically perform self-healing operations such as Pod rescheduling, reducing manual intervention and ensuring high availability of core financial applications in enterprises. Finally, by introducing reinforcement learning to optimize the commit strategy in distributed transactions, the present application not only guarantees strong consistency of financial data, but also intelligently reduces transaction lock competition and reduces processing delay, thereby improving overall transaction processing efficiency in high-concurrency scenarios. In summary, the present application not only realizes deep collaboration at the deployment level, but also integrates artificial intelligence technology to create an efficient, intelligent, reliable and self-adaptive industry and finance integration platform with outstanding technical advantages and application value. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1This is a flowchart illustrating the collaborative method for containerized hybrid deployment of database and financial software provided in a specific embodiment of the present invention. Figure 2 This is a schematic diagram of the system composition of a collaborative system for containerized hybrid deployment of database and financial software provided in a specific embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments in this application, other similar embodiments obtained by those skilled in the art without creative effort should all fall within the scope of protection of this application. Furthermore, directional terms mentioned in the following embodiments, such as "up," "down," "left," and "right," are only for reference to the directions in the accompanying drawings; therefore, the directional terms used are for illustrative purposes and not for limiting the invention.

[0018] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0019] Please see Figure 1 This invention provides a collaborative method for containerized hybrid deployment of databases and financial software, comprising: Step S1: Package the database and financial software into separate containers and deploy them in a hybrid manner on a container orchestration platform to form a database container and a financial software container.

[0020] It should be further explained that this invention provides a fundamental technical framework for the deep integration of business and financial systems. The specific implementation of step S1 is as follows: The database is preferably MySQL 8.0 or PostgreSQL 14, and the financial software is typically represented by Kingdee EAS or Yonyou NC. When packaging the container, a multi-layered image build is defined using Dockerfile: the base layer uses Alpine Linux 3.18 to reduce image size; the middle layer installs the runtime environment (such as Java 17, Python 3.9); the application layer deploys the database or financial software binary files and configuration files. After the image is built, it is pushed to a private image repository (such as Harbor). The container orchestration platform uses Kube containers to share the same availability zone, and sets the initial resource requests and limits through the resource field. For example, the database container is configured with a CPU request of 2 cores, a limit of 4 cores, and a memory request of 4GB, a limit of 8GB; the financial software container is configured with a CPU request of 1 core, a limit of 2 cores, and a memory request of 2GB, a limit of 4GB.

[0021] Step S2: Establish a collaborative communication mechanism between the database container and the financial software container to perform data synchronization and dynamic collaborative allocation of resources.

[0022] It should be further explained that the collaborative communication mechanism in step S2 is specifically implemented as follows: Data synchronization adopts a real-time capture technology based on transaction logs (such as Debezium), monitors changes in the database bin log and converts them into Avro format event messages, which are then asynchronously pushed to the financial software container via a Kafka message queue to ensure eventual consistency. Dynamic collaborative resource allocation is implemented through a custom Kubernetes controller. This controller queries the Metrics Server every 30 seconds to obtain the real-time resource utilization of each Pod. When it detects that the CPU utilization of the database container is higher than 75% for 2 consecutive minutes, it automatically dynamically increases the CPU limit of the database container by 0.5 cores through the VPA (VerticalPod Autoscaler) API, and simultaneously adjusts the resource quota of the associated financial software container proportionally to maintain balance.

[0023] Step S3: Based on a deep learning model, analyze business data and system load in real time, and dynamically optimize container resource allocation and data access strategies.

[0024] It should be further explained that the deep learning model integrated in step S3 is a multivariate time series prediction model with 8-dimensional input features, including business data (number of transactions per minute, average transaction amount, concurrent users) and system load data (CPU utilization, memory utilization, network throughput, disk IOPS, P99 latency). The model uses an encoder-decoder architecture LSTM network, with both the encoder and decoder containing two LSTM layers, each with 64 neurons and 128 hidden states. The model uses historical data from the past 60 minutes (sliding window size of 60) to predict the load trend for the next 15 minutes. Training uses the TensorFlow 2.13 framework, with Adam W as the optimizer, an initial learning rate of 0.001, decaying to 0.8 times the original rate every 50 epochs, a batch size of 32, 200 training epochs, and an early stopping patience value of 15. The loss function is Huber loss, with the following formula: in, Represents the true value. Represents the model's predicted value. This is the threshold parameter, with a preferred value of 1.0. The reason for choosing 8=1.0 is that, in load forecasting scenarios, this value strikes a good balance between the high sensitivity of mean squared error (MSE) to outliers and the gradient stability of mean absolute error (MAE). The predicted values ​​output by the model are used to trigger elastic scaling: when the predicted CPU utilization exceeds the 85% threshold and continues to predict for 5 minutes, the Kubernetes API is invoked to increase the number of replicas of the financial software container; when the predicted value is below 40%, the number of replicas is reduced. The choice of the 85% threshold is based on long-term operational experience; this level usually indicates that the system is about to enter the performance saturation zone, and scaling up 5 minutes in advance can avoid response time degradation; the 40% threshold ensures that resources are not prematurely reclaimed, leading to a decrease in the ability to handle sudden traffic.

[0025] Understandably, this approach achieves deep coupling between the business and finance systems at the deployment level through containerized isolation and orchestration, real-time data synchronization mechanisms, and deep learning-based predictive scaling. This effectively breaks down data silos and solves performance bottlenecks. The introduction of deep learning transforms the system from a passive response to an active planning approach, significantly improving resource utilization efficiency and business continuity.

[0026] Specifically, the container orchestration platform in step S1 is based on Kubernetes, and the database container and financial software container are deployed in groups through the Kubernetes Pod mechanism to ensure locality optimization.

[0027] Further explanation is needed. In the specific implementation, a Pod named "Business & Finance Unit" is created, containing two containers: a database container (e.g., MySQL) and a financial software container (e.g., yonyou-nc). Pod-level configuration allows the two containers to share the same network namespace (they can communicate directly via local hosts, using ports 3306 and 8080 respectively) and storage volume (an empty Dir volume named shared-data used for temporary file exchange). To ensure locality of reference optimization, node affinity rules are defined in the Pod configuration, requiring the Pod to be scheduled to nodes with the labels disktype=ssd and zone=primary. Resource requests (requests.cpu: 3, requests.memory: 6Gi) and limits (limits.cpu: 6, limits.memory: 12Gi) are set for the Pod so that the Kubernetes scheduler can make optimal node selections. Additionally, probes (liveness probe, readiness probe) are configured for the Pod to ensure rapid restart or service shutdown in case of container anomalies.

[0028] Understandably, by deploying Pods in groups, the communication latency between the database and the financial software is reduced to sub-millisecond levels, avoiding cross-node network calls, greatly improving data access speed, achieving true locality optimization, and laying the foundation for high-performance business and financial processing.

[0029] Specifically, the collaborative communication mechanism in step S2 adopts service mesh technology to establish a two-way certified gRPC channel between the database container and the financial software container to achieve low-latency data exchange.

[0030] It should be further explained that this invention provides an implementation scheme for service mesh and gRPC channels. The service mesh technology used is Istio 1.19. An Istio control plane is deployed in a Kubernetes cluster, and Envoy sidecar proxies are automatically injected into Pods containing database and financial software containers. The process of establishing a two-way authenticated gRPC channel is as follows: First, namespace-level STRICT mode mTLS (two-way TLS) is set using Istio's Peer Authentication policy. Second, Destination Rules are created for the database service and financial software service respectively, defining a secure traffic policy using the ISTIO_MUTUALTLS mode. The gRPC server (within the database container) is written in Go, implementing predefined proto file interfaces (such as the Data Sync service), while the client (within the financial software container) stub code is generated by the protoc compiler. During communication, the client initiates a connection via the service name (e.g., mysql-service.finance-namespace.svc.cluster.local:3306), and the Envoy proxy completes service discovery, load balancing, and the mTLS handshake. To optimize low latency, gRPC channel parameters were adjusted: grpc.keepalive_time_ms was set to 10000 milliseconds (10 seconds), and grpc.http2.max_pings_without_data was set to 0, allowing unconditional ping frame keep-alive transmission. Data serialization uses Protocol Buffers, which improves serialization / deserialization speed by approximately 3-5 times compared to JSON.

[0031] Understandably, the service mesh provides a unified and secure communication layer, gRPC ensures efficient and reliable binary data exchange, and combined with mTLS two-way authentication, it guarantees the security and low latency of financial data transmission in complex network environments (P99 latency can be controlled within 10 milliseconds).

[0032] Specifically, the deep learning model in step S3 is a Long Short-Term Memory (LSTM) network, which is used to predict resource demand during peak business periods and trigger container autoscaling in advance.

[0033] It should be further noted that the input of this LSTM model is a tensor of shape (batch_size, sequence_length, input_dim), where the preferred value for sequence_length (time step) is 60, representing the data from the past 60 minutes; and the input_dim (feature dimension) is 8. The model architecture is as follows: ① Input layer: Receives input of type (None, 60, 8); ② First LSTM layer: 64 neurons, returns complete sequence (return_sequences=True); ③ Second LSTM layer: 32 neurons, only returns the output of the last time step (return_sequences=False); ④ Dropout layer: The dropout rate is set to 0.3 to prevent overfitting; ⑤ Fully connected layer (Dense): 16 neurons, activation function is ReLU; ⑥ Output layer (Dense): 1 neuron, activation function is a linear function, predicts CPU usage for the next 15 minutes (normalized to between 0 and 1).

[0034] Furthermore, before model training, the data is Z-score standardized. Training uses the Huber loss function, with Adam W as the optimizer and a weight decay rate of 0.01. The learning rate is scheduled using cosine annealing decay, as shown in the following formula: in: : The actual learning rate used at step t (or epoch t) of training. This is the result of the formula calculation; The preset minimum learning rate lower limit, with an optimal value of 1×10. -5 The preferred reason is that setting a non-zero minimum value can prevent the model from stagnating due to excessively small update steps in the later stages of training. At the same time, this value is small enough to ensure that the model stabilizes when it approaches the optimal point. The preset maximum learning rate limit, with an optimal value of 1×10. -3The preferred reason is that this value is a commonly used large initial learning rate, which helps to decrease quickly in the early stages of training. At the same time, for optimizers such as Adam W, this value is usually within a stable range and is less likely to cause gradient explosion. The current epoch number represents the number of training iterations that have been performed. Total number of training epochs, with an optimal value of 200. This setting is based on the complexity of the specific task (such as resource prediction) to ensure that the model has enough epochs to fully learn the data features, while avoiding excessively long training time. The period of the cosine function is defined by this value.

[0035] Understandably, choosing a cosine annealing strategy allows the learning rate to smoothly decrease from its maximum to its minimum as the training progresses. This smooth descent characteristic helps the model converge quickly with large steps in the early stages of training and finely adjust with small steps in the later stages, thus making it more likely to converge to a better local optimum and avoiding training instability that may be caused by sudden changes in the learning rate.

[0036] Furthermore, after the model is integrated into the system, a prediction is performed every 5 minutes. If the CPU utilization exceeds 85% at any point within the predicted next 15 minutes, HPA (Horizontal Pod Auto Scaler) is triggered for horizontal scaling, with the target CPU utilization set at 75%. For example, if a peak is predicted, HPA will increase the number of replicas of the financial software container from 3 to 5. 85% is chosen as the prediction trigger threshold because it is slightly higher than HPA's target value of 75%, allowing sufficient time for the scaling operation (which typically takes 1-2 minutes), achieving true "early" triggering.

[0037] Understandably, the LSTM model accurately captures complex time dependencies, and predictive scaling avoids the lag of reactive scaling, ensuring smooth system operation and stable response performance during peak business periods.

[0038] Specifically, the dynamic collaborative allocation of resources in step S2 includes: dynamically adjusting the CPU and memory resource quotas of the database container and the financial software container based on real-time monitoring indicators to achieve load balancing.

[0039] It should be further explained that this invention provides specific strategies for dynamic resource adjustment. Real-time monitoring metrics are collected by Prometheus 2.45 deployed in the cluster, using Node Exporter and cAdvisor to capture container-level metrics, with a sampling interval of 15 seconds. The core logic for dynamic adjustment is implemented by a custom Kubernetes Operator, which continuously monitors Prometheus metrics. The adjustment strategy is based on the following rules: 1. CPU tuning rules: Calculate the average CPU utilization of the database container over the past 5 minutes (CPU). avg If (CPU) avg If CPU usage exceeds 80%, both its CPU request and limit will be increased by 0.25 cores, but the limit must not exceed the preset upper limit (e.g., 8 cores). Simultaneously, to maintain balance, the CPU limit of the associated financial software container will be temporarily reduced by 0.1 cores (but not lower than 0.5 cores) to avoid node resource contention. If (CPU...) avg If the percentage is less than 40% for 10 minutes, then reverse the operation. 2. Memory Tuning Rules: Calculate memory utilization (MEM) avg ) and Working Set Size. If (MEM avg If the memory requirement is greater than 85% and the working set size continues to grow, increase the memory request and limit by 512 MiB. Memory adjustments should be made more cautiously, with a 10-minute observation period after each adjustment to ensure there is no memory pressure before proceeding with further operations.

[0040] Understandably, the 80% CPU threshold prevents the system from entering the CPU scheduling bottleneck area; the 85% memory threshold ensures a buffer before memory pressure surges, preventing Out-of-Memory (OOM) errors. Adjusting the step size to 0.25 cores and 512 MiB is based on test results of sensitivity to typical workloads, effectively alleviating pressure without causing drastic fluctuations in resource allocation.

[0041] Understandably, this enables fine-grained and interconnected management of container-level resources, ensuring that the two critical workloads—database and financial software—always have access to the necessary computing resources, achieving cross-application load balancing, and improving the overall cluster's resource utilization.

[0042] Specifically, the real-time monitoring metrics include container-level CPU utilization, memory utilization, network I / O, and disk I / O, which are collected and stored through the Prometheus system.

[0043] It should be further explained that this invention achieves fine-grained monitoring of container resources through the Prometheus monitoring system and its component cAdvisor deployed in a Kubernetes cluster. The specific calculation methods, collection details, and parameter settings of each monitoring metric are described below. These metrics form the data foundation for subsequent dynamic resource adjustments and intelligent optimizations: ① CPU Utilization: CPU utilization is a core metric for measuring the computational resource pressure on containers. cAdvisor calculates it by reading the Linux kernel's CPU statistics (located in the / sys / fs / cgroup / cpuacct / directory). Its standardized calculation formula is as follows: in: : Represents the increment of CPU time actually consumed by the container process within a sampling interval. cpu_usage_total is a cumulative value (usually in nanoseconds). This represents the difference between two consecutive samples of this value; The length of a scheduling cycle (in microseconds) of CFS (Completely Fair Scheduler) is a system-level parameter, usually fixed at 100,000 microseconds (i.e. 100 milliseconds). Within a cpu_period, the maximum CPU time (in microseconds) that the container is allowed to use. For example, if the container's CPU is limited to 1 core, then cpu_quota is 100,000 microseconds; if it is limited to 0.5 cores, then it is 50,000 microseconds. As is understandable, this formula calculates the percentage of CPU time actually used by a container relative to its maximum allowed CPU time within a sampling period. This accurately reflects the saturation level of the container's CPU resources.

[0044] ② Memory Usage: To accurately assess memory pressure and avoid misjudgments due to heavy caching by the Linux system, this invention does not directly use total memory usage, but instead uses a working set of memory as the evaluation criterion. The calculation method is as follows: in: Total memory usage of the container (in bytes). This includes application code, heap memory, stack memory, and memory used by the cache. The size of memory occupied by reclaimable caches such as page cache within the container (in bytes). This memory can be quickly discarded by the kernel when system memory is insufficient, therefore it does not constitute a real memory pressure. The calculation results reflect the amount of non-reclaimable memory that a container must reside in physical memory to maintain its current running state. It is a key indicator for assessing real memory pressure and determining whether an Out-of-Memory (OOM) killer will be triggered, and is more valuable than total usage.

[0045] ③ Network I / O: Network I / O metrics are used to monitor the network bandwidth usage of containers.

[0046] (1) Metrics: network_receive_bytes_total (cumulative number of bytes received), network_transmit_bytes_total (cumulative number of bytes sent); (2) Calculation method: Prometheus calculates the network throughput (unit: bytes / second) by calculating the difference between these counter metrics within the sampling interval; the formula is: ④ Disk I / O: Disk I / O metrics reflect the read and write pressure of the container on block devices.

[0047] (1) Indicator Items: ocontainer_fs_reads_total, container_fs_writes_total: Total number of read and write operations, used to calculate IOPS.

[0048] ocontainer_fs_read_bytes_total, container_fs_write_bytes_total: Total number of bytes read and written, used to calculate throughput.

[0049] (2) Calculation method: Similarly, by calculating the difference and dividing by the sampling interval, the number of read / write operations per second and the number of read / write bytes per second are obtained.

[0050] Furthermore, regarding the Prometheus capture and storage configuration, to ensure the real-time nature and traceability of monitoring data, this invention makes the following key configurations for Prometheus: (1) Scraping interval: In Prometheus's scrape_config, scrape_interval is set to 15 seconds. It is understandable that a 15-second interval is a common value in the industry for balancing monitoring accuracy and Prometheus server load. Too long an interval (such as 1 minute) may not be able to capture instantaneous traffic spikes, while too short an interval (such as 5 seconds) will bring unnecessary overhead to the monitoring system and network; (2) Data retention period: The retention period of the time series database is set to 30 days. It can be understood that a 30-day retention period is sufficient to cover a complete financial monthly cycle, providing sufficient historical data support for capacity planning, fault diagnosis and deep learning model training. (3) Recording rules: To reduce the real-time computing load during queries, recording rules are pre-configured to pre-aggregate high-frequency raw metrics. For example, the rule for calculating the average CPU utilization over 5 minutes is: "record: container_cpu_usage_5m_avg" expr: avg_over_time(container_cpu_usage_seconds_total[5m])"; The avg_over_time function calculates the average value of all points within a given time window ([5m]) for a specified time series.

[0051] Understandably, the application directly queries the pre-calculated metric container_cpu_usage_5m_avg, avoiding the need to perform aggregation calculations on large amounts of raw data each time, which greatly improves the response speed of the monitoring dashboard and alarm rules.

[0052] Understandably, Prometheus enables comprehensive and high-frequency monitoring of container resources, and the collected metrics are clearly defined and accurately calculated, providing a reliable data foundation for subsequent dynamic resource adjustments and intelligent decision-making.

[0053] Specifically, this also includes: triggering an early warning mechanism when real-time monitoring metrics exceed preset thresholds, and automatically rescheduling container instances by using a deep learning model for anomaly detection and self-healing.

[0054] It should be further noted that this invention adds an abnormal self-healing capability. The preset thresholds for the early warning mechanism are as follows: ① CPU utilization: Warning threshold 85%, critical threshold 95%; ② Memory usage: Warning threshold 90%, critical threshold 98%; ③ Disk space usage: Warning threshold 80%, critical threshold 90%.

[0055] Furthermore, when Prometheus detects that an indicator exceeds the warning threshold, the Alert Manager will send an alert to DingTalk or email according to the configured routing rules. Anomaly detection employs an autoencoder model based on unsupervised learning. The model's input is an 8-dimensional monitoring indicator (same as the LSTM input) from approximately 10 sampling points (i.e., 2.5 minutes), and its goal is to learn the data distribution under normal conditions. The model structure is a symmetric encoder-decoder, with the encoder consisting of three fully connected layers (64, 32, and 16 neurons respectively), and the decoder being symmetric (16->32->64), with a bottleneck layer dimension of 8. Reconstruction error is calculated using mean squared error (MSE). The MSE threshold is determined by calculating the 99th percentile of the reconstruction error of normal samples on the validation set, assumed to be 0.05. When the reconstruction error MSE of real-time data > 0.05, it is considered an anomaly.

[0056] Furthermore, the self-healing operation is performed by a self-developed Kubernetes controller: First, the controller listens for Prometheus alerts and abnormal outputs from the autoencoder; once an anomaly is confirmed (e.g., three consecutive sample points are abnormal), a node-selector label is automatically added to the abnormal Pod, and the Pod is evicted; the Kubernetes scheduler then reschedules the Pod to a healthy node in the cluster that meets the label requirements.

[0057] Understandably, combining threshold warnings and deep learning anomaly detection can promptly identify potential problems such as hardware failures and software deadlocks, and achieve fault self-healing through automated Pod rescheduling, greatly improving the robustness and availability of the system and reducing manual intervention in operations and maintenance.

[0058] Specifically, the data synchronization in step S2 adopts a distributed transaction protocol to ensure strong consistency of financial data transmitted between containers.

[0059] It should be further explained that this invention explicitly guarantees strong consistency of data synchronization. Specifically, a two-phase commit (2PC) protocol is adopted. In a distributed transaction scenario, the database container acts as the transaction coordinator (TC), and the financial software container acts as the participant.

[0060] Furthermore, the protocol execution steps are as follows: 1. Preparation Phase: The financial software container (participant) initiates an update request to the database container (coordinator). The coordinator sends a PREPARE message to all participants. Upon receiving the message, each participant executes the transaction operation until completion (writing to the Redo Log), but does not commit. They then reply to the coordinator with either YES (indicating they are ready to commit) or NO (indicating they cannot complete preparation). 2. Commit Phase: If the coordinator receives YES responses from all participants, it decides to commit the transaction and sends a COMMIT message to all participants. Upon receiving a COMMIT message, the participants formally commit the transaction. If the coordinator receives a NO response from any participant or a timeout occurs, it sends a ROLLBACK message to roll back the transaction.

[0061] Furthermore, to improve reliability, the coordinator persists transaction decisions to a log. The timeout setting in the protocol is crucial: a 30-second timeout for the preparation phase and a 60-second timeout for the commit phase. The 30-second timeout is chosen to account for the complexity of financial transactions and network latency, while the 60-second timeout provides ample buffering for potential disk I / O during the commit phase.

[0062] Understandably, the 2PC protocol ensures that, in a distributed container environment, the update operations of financial data meet the atomicity and consistency characteristics of ACID, preventing data inconsistencies and fully complying with the stringent requirements of financial systems for data accuracy.

[0063] Specifically, the distributed transaction protocol is implemented based on two-phase commit (2PC) and the transaction commit strategy is optimized through a deep learning model to reduce lock contention and latency.

[0064] It should be further explained that this invention utilizes deep learning to optimize 2PC performance. The optimization goal is to reduce the "blocking" problem caused by coordinator failure or network partitioning after the preparation phase. A deep reinforcement learning (DRL) model is employed, specifically the proximal policy optimization (PPO) algorithm.

[0065] Furthermore, its elements are defined as follows: ①State space: includes the number of currently active transactions, the number of locks held by each transaction, the average execution time of transactions, and the current system load.

[0066] ②Action Space: After the coordinator receives YES from all participants, it can choose to commit immediately (COMMIT) or delay committing (DELAY_COMMIT, wait a short time before committing to avoid peak lock-breaking competition).

[0067] ③ Reward function: in: Reward value is the immediate feedback that an agent receives after making a decision. The goal of an agent is to maximize long-term cumulative rewards. The system's transaction throughput typically refers to the number of transactions successfully processed per unit of time. Agents should strive to improve this value. The average processing latency of transactions; agents should strive to reduce this value. The number of currently blocked transactions. This value directly reflects the severity of resource contention and lock conflicts within the system. Agents should prioritize reducing this value. : The weighting coefficients for the corresponding performance indicators. Their preferred values ​​are set to 1.0, 0.5, and 2.0, respectively.

[0068] Understandably, the reasons for setting optimal weight coefficient values ​​include: the weight assignment clearly reflects the priority of the optimization objective. (Number of blocked transactions) Setting it to the highest value (2.0) is to emphasize the importance of reducing blocking. In database concurrency control, transaction blocking is a major factor leading to performance bottlenecks and latency spikes. Prioritizing the mitigation of blocking can usually more effectively improve overall system performance (i.e., simultaneously improve throughput). and delay ). The second (1.0) indicates that the ultimate goal is to increase throughput. The minimum value (0.5) is chosen because reducing latency is typically an improvement that occurs alongside reducing congestion and increasing throughput. This weighting guides the agent to learn strategies that prioritize addressing key bottlenecks.

[0069] Furthermore, the PPO algorithm uses an Actor-Critic architecture, where both the Actor and Critic networks are two-layer fully connected networks (128 neurons each). The model is trained in a simulated environment to learn the optimal commit strategy. For example, the model may learn to adopt a delayed commit strategy for non-critical transactions under high system load and intense lock contention, thereby reducing the overall blocking probability.

[0070] Understandably, by dynamically optimizing the decision points of 2PC through the DRL model, consistency requirements and performance overhead are intelligently balanced, significantly reducing transaction lock contention and average commit latency (which can be reduced by about 15% in the experimental environment), and improving the processing efficiency of high-concurrency financial transactions.

[0071] Please see Figure 2The present invention provides another embodiment, which provides a collaborative system for the containerized hybrid deployment of database and financial software. The collaborative system for the containerized hybrid deployment of database and financial software includes: The containerized deployment module 100 is used to encapsulate the database and financial software into independent containers and perform hybrid deployment on a container orchestration platform. It should be further explained that the containerized deployment module 100 is a CI / CD pipeline based on Jenkins. After developers submit code, the pipeline automatically executes `docker build` to build the image and calls `kubectl apply` to deploy it in the Kubernetes cluster. This module also includes a graphical console (based on Rancher or Kube Sphere) for visually defining and maintaining the deployment topology and resource constraints of the database and financial software.

[0072] The collaborative communication module 200 is used to establish a data synchronization and dynamic resource collaborative allocation mechanism between the database container and the financial software container; Further explanation is needed regarding the collaborative communication module 200: This module consists of several Kubernetes custom resource definitions (CRDs) and their corresponding controllers. For example, the Data Sync CRD is used to define data synchronization rules (such as source table, target table, and synchronization frequency), and its controller is responsible for deploying and monitoring the Debezium Connector. The ResourceSync CRD is used to define resource collaboration policies, and its controller is responsible for listening to the Metrics API and calling the VPA / HPA API for scaling.

[0073] The Deep Learning Optimization Module 300 integrates deep learning models for real-time analysis of business data and system load, and dynamically adjusts container resource allocation and data access strategies. It should be further noted that the deep learning optimization module 300 is deployed as a separate Kubernetes Deployment, containing three containers: 1) Model-Serving container: Runs model services based on TensorFlow Serving and provides a RESTful API for other modules to query prediction results.

[0074] 2) Model-Training container: Regularly (e.g., every morning) pulls the latest data from Prometheus, retrains the LSTM and autoencoder models, and verifies the model performance. If the performance improves, the production model is automatically updated.

[0075] 3) DRL-Agent container: Runs the PPO agent, interacts with the simulation environment, and continuously optimizes the 2PC strategy.

[0076] The system directly responds to the need for "business-finance integration" and solves the problems of data silos and performance bottlenecks.

[0077] It should be further noted that the system is deployed on a Kubernetes cluster consisting of at least 3 nodes (1 Master and 2 Workers). The modules of the system communicate with each other through Kubernetes Service names, and the data flow is clear.

[0078] Understandably, this system organically integrates modern cloud-native and AI technologies such as container orchestration, service mesh, monitoring and alerting, and deep learning into a complete solution. It achieves automated and intelligent management of the entire lifecycle of the business and finance system, from deployment and communication to optimization, effectively solving data silos and performance bottlenecks, and providing a solid and reliable technical platform for "business and finance integration".

[0079] In a preferred embodiment, this application also provides an electronic device, the electronic device comprising: The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the collaborative method for containerized hybrid deployment of the database and financial software. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include non-volatile storage media and internal memory. The non-volatile storage media may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the method of the present invention.

[0080] This invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0081] Those skilled in the art will understand that the method steps of this invention can be performed by a computer program instructing related hardware, such as a computer device or processor, to perform the steps of this invention when executed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0082] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0083] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A collaborative method for containerized hybrid deployment of databases and financial software, characterized in that, Includes the following steps: S1. Encapsulate the database and financial software into separate containers and deploy them in a hybrid manner on a container orchestration platform to form a database container and a financial software container; S2. Establish a collaborative communication mechanism between the database container and the financial software container to perform data synchronization and dynamic collaborative allocation of resources. S3, based on a deep learning model, analyzes business data and system load in real time, and dynamically optimizes container resource allocation and data access strategies.

2. The collaborative method for containerized hybrid deployment of database and financial software according to claim 1, characterized in that, The container orchestration platform in step S1 is based on Kubernetes, and the database container and financial software container are deployed in groups through the Kubernetes Pod mechanism.

3. The collaborative method for containerized hybrid deployment of database and financial software according to claim 2, characterized in that, The collaborative communication mechanism in step S2 adopts service mesh technology to establish a two-way certified gRPC channel between the database container and the financial software container.

4. The collaborative method for containerized hybrid deployment of database and financial software according to claim 3, characterized in that, The deep learning model in step S3 is a Long Short-Term Memory (LSTM) network, which is used to predict resource requirements during peak business periods and trigger container auto-scaling operations in advance.

5. The collaborative method for containerized hybrid deployment of database and financial software according to claim 1, characterized in that, The dynamic collaborative allocation of resources in step S2 includes: Based on real-time monitoring metrics, dynamically adjust the CPU and memory resource quotas for database containers and financial software containers.

6. The collaborative method for containerized hybrid deployment of database and financial software according to claim 5, characterized in that, The real-time monitoring metrics include container-level CPU utilization, memory utilization, network I / O, and disk I / O, which are collected and stored through the Prometheus system.

7. The collaborative method for containerized hybrid deployment of database and financial software according to claim 6, characterized in that, Also includes: When real-time monitoring metrics exceed preset thresholds, an early warning mechanism is triggered, and anomaly detection and self-healing operations are performed through a deep learning model, automatically rescheduling container instances.

8. The collaborative method for containerized hybrid deployment of database and financial software according to claim 1, characterized in that, The data synchronization in step S2 adopts a distributed transaction protocol.

9. The collaborative method for containerized hybrid deployment of database and financial software according to claim 8, characterized in that, The distributed transaction protocol is implemented based on two-phase commit and the transaction commit strategy is optimized through a deep learning model.

10. A collaborative system for containerized hybrid deployment of database and financial software, characterized in that, include: The containerized deployment module is used to encapsulate databases and financial software into independent containers and perform hybrid deployment on a container orchestration platform; The collaborative communication module is used to establish a data synchronization and dynamic resource allocation mechanism between the database container and the financial software container. The deep learning optimization module integrates deep learning models for real-time analysis of business data and system load, and dynamically adjusts container resource allocation and data access strategies.

Images