Containerized resource dynamic allocation method for power distribution network system and electronic equipment

Abstract

The invention discloses a containerized resource dynamic allocation method for a power distribution network system and electronic equipment. The method comprises the following steps: judging that resource use data is greater than a preset resource threshold value by monitoring the resource use condition of each physical node in a power distribution network system; if yes, calculating a load balancing index of each physical node, and generating a container migration task list based on the load balancing index, the service priority and the service quality requirement information; packaging container data corresponding to the to-be-migrated container in the migration task list, and determining container state snapshot information and network connection information of the to-be-migrated container; establishing a secure channel between the source node and the target node based on the network connection information, and transmitting container data and container state snapshot information of the source node to the target node; reconstructing a container operation environment on the target node, and generating a target container; and suspending the to-be-migrated container on the source node, and activating the target container on the target node. And efficient migration of the container is realized.

Description

Technical Field

[0001] The present invention relates to the field of power distribution technology, and in particular to a containerized dynamic resource allocation method and electronic equipment for a power distribution network system. Background Art

[0002] In the power distribution network system, dynamic allocation of containerized resources faces a complex technical challenge. When the system needs to reallocate resources across physical nodes, how to achieve flexible resource scheduling while ensuring business continuity becomes a key issue. Specifically, when performing hot migration of containers, it is necessary to consider multi-dimensional virtualization encapsulation, including physical attributes, logical attributes, and geographic location. This multi-dimensional encapsulation increases the complexity of the migration process, especially in cross-region or cross-data center scenarios. At the same time, the dynamic adjustment of resource pools also brings new challenges.

[0003] How to ensure data consistency and integrity during resource displacement and migration is also a thorny issue. Especially in the case of large-scale data migration, how to minimize the impact of data transmission on network bandwidth while ensuring the reliability and efficiency of the migration process has become a technical problem that needs to be solved urgently. The core of these challenges lies in how to achieve accurate resource allocation and smooth migration in a dynamically changing environment while ensuring high system availability and business continuity. This requires in-depth research and innovation in multiple fields such as container technology, virtualization technology, network technology, and scheduling algorithms to develop more intelligent and efficient resource management solutions. Summary of the invention

[0004] The present invention provides a containerized resource dynamic allocation method for a power distribution network system and an electronic device to achieve accurate allocation of power distribution network system resources and smooth migration of containers, thereby ensuring high availability of the system and continuity of business.

[0005] According to one aspect of the present invention, a containerized resource dynamic allocation method for a power distribution network system is provided, comprising:

[0006] The resource usage status of each physical node in the power distribution network system is monitored in real time through the container resource monitoring module to obtain resource usage data of each physical node, and determine whether there is any physical node whose resource usage data is greater than a preset resource threshold;

[0007] If the resource usage data of the physical node exists and is greater than the preset resource threshold, the overall resource distribution status of the power distribution network system is analyzed through the resource evaluation algorithm, the load balancing index of each physical node is calculated, and based on the load balancing index, business priority and service quality requirement information corresponding to the physical node, a container migration task list is generated; wherein the container migration task list includes a source node, a target node, a container identifier and a migration time window;

[0008] According to the container migration task list, calling the container migration preprocessing module, encapsulating the container data corresponding to the container to be migrated in the migration task list, and determining the container state snapshot information and network connection information of the container to be migrated;

[0009] Starting a cross-node data transmission module, establishing a secure channel between the source node and the target node based on the network connection information, using an incremental transmission algorithm to transmit the container data and the container status snapshot information of the source node to the target node, and verifying them through a data verification mechanism during the transmission process;

[0010] After completing the verification, the calling container reconstruction module on the target node is started, and the container operating environment is rebuilt according to the container data and the container status snapshot information, and the target container is generated;

[0011] The container to be migrated on the source node is paused, and the target container on the target node is activated, the routing table and load balancing configuration of the power distribution network system are updated, and the service traffic of the container to be migrated is switched to the target container.

[0012] According to another aspect of the present invention, a containerized resource dynamic allocation device for a power distribution network system is provided, comprising:

[0013] A resource usage judgment module is used to monitor the resource usage status of each physical node in the power distribution network system in real time through the container resource monitoring module, obtain the resource usage data of each physical node, and judge whether there is a physical node whose resource usage data is greater than a preset resource threshold;

[0014] A migration list generation module is used to analyze the overall resource distribution status of the power distribution network system through a resource evaluation algorithm if the resource usage data of the physical node is greater than a preset resource threshold, calculate the load balancing index of each physical node, and generate a container migration task list based on the load balancing index, business priority and service quality requirement information corresponding to the physical node; wherein the container migration task list includes a source node, a target node, a container identifier and a migration time window;

[0015] A container encapsulation module, configured to call a container migration preprocessing module according to a container migration task list, encapsulate container data corresponding to the container to be migrated in the migration task list, and determine container state snapshot information and network connection information of the container to be migrated;

[0016] A data transmission module, used to start a cross-node data transmission module, establish a secure channel between the source node and the target node based on the network connection information, adopt an incremental transmission algorithm to transmit the container data and the container status snapshot information of the source node to the target node, and verify them through a data verification mechanism during the transmission process;

[0017] A container reconstruction module, used to start the calling container reconstruction module on the target node after completing the verification, rebuild the container operating environment according to the container data and the container status snapshot information, and generate a target container;

[0018] The container migration module is used to pause the container to be migrated on the source node, activate the target container on the target node, update the routing table and load balancing configuration of the power distribution network system, and switch the service traffic of the container to be migrated to the target container.

[0019] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0020] at least one processor;

[0021] and a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the containerized dynamic resource allocation method for a distribution network system described in any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for dynamically allocating resources for containerized distribution network systems described in any embodiment of the present invention when executed.

[0024] The technical solution of the embodiment of the present invention first combines business priorities and service quality requirements to generate the optimal migration plan, and uses incremental transmission and state snapshot technology to achieve efficient migration of containers, thereby improving the efficiency of migration and reducing the complexity of migration; in addition, during the migration process, resource isolation is used to ensure the consistency of the reconstructed container, and a fast fault detection and rollback mechanism is used to ensure business continuity, ensure the reliability of the migration process, and ensure the integrity of the migration data; finally, through dynamic resource scheduling and load balancing, it is ensured that the new container can quickly and accurately build the operating environment, solve the complexity problem of the target node container reconstruction, realize flexible allocation of resources, can quickly adapt to business changes and load fluctuations, reduce system maintenance costs, and reduce manual intervention and errors through automated resource recovery and status updates. In summary, this method realizes the intelligent, automated and dynamic allocation of distribution network system resources, improves the system's reliability and resource utilization, and enhances the system's elasticity and reliability.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A flow chart of a containerized resource dynamic allocation method for a power distribution network system provided in Embodiment 1 of the present invention;

[0028] Figure 2 A flowchart of cross-node data transmission provided by Embodiment 1 of the present invention;

[0029] Figure 3 A flowchart of resource recycling provided in Embodiment 1 of the present invention;

[0030] Figure 4 A schematic diagram of the structure of a containerized resource dynamic allocation device for a power distribution network system provided in the second embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] Figure 1 This is a flowchart of a method for dynamically allocating resources in a containerized power distribution network system provided in the first embodiment of the present invention. This embodiment can be applied to the case of performing container migration for a power distribution network system and intelligently and efficiently allocating resources in the power distribution network system. This method can be executed by a dynamic resource allocation device for a containerized power distribution network system. The device can be implemented in the form of hardware and / or software. The device can be configured in a power distribution network system, and the power distribution network system can be composed of multiple servers. Figure 1 As shown, the method includes:

[0036] S110. Monitor the resource usage status of each physical node in the power distribution network system in real time through the container resource monitoring module, obtain the resource usage data of each physical node, and determine whether there is a physical node whose resource usage data is greater than a preset resource threshold.

[0037] Among them, the distribution network system provides a service cluster of computing, storage and other resources for the distribution network. Each server in the cluster is a physical node. The corresponding resource management system and container can be deployed on the server. A large number of businesses in the distribution network can be processed through the container; the resource usage data can be data reflecting the CPU utilization, memory usage and network bandwidth of each physical node; the preset resource threshold can be a pre-set value.

[0038] Specifically, a container resource monitoring module is used to monitor the resource usage status of each physical node in the power distribution network system in real time, collect multi-dimensional data including CPU utilization, memory usage and network bandwidth, and determine whether to trigger the resource reallocation mechanism based on the preset resource threshold. When it is detected that the resources of a certain node exceed the threshold, it means that the resources of the power distribution network system need to be reallocated. The container migration evaluation process can be started to migrate some containers from physical nodes with insufficient resources to physical nodes with sufficient resources.

[0039] In an embodiment of the present invention, the resource usage status of each physical node in the distribution network system is monitored in real time through the container resource monitoring module to obtain the resource usage data of each physical node, and to determine whether there is any physical node whose resource usage data is greater than a preset resource threshold, including: for the distribution network system, using a distributed monitoring agent program to monitor the resource usage status of each physical node in real time to obtain performance indicator data; based on the performance indicator data, performing statistical analysis on each performance indicator to calculate the resource usage data, the resource usage data including the resource usage mean, peak value and variance; determining whether the resource usage data is greater than a preset resource threshold.

[0040] The performance indicator data includes CPU utilization, memory usage, disk I / O and network bandwidth.

[0041] Specifically, a distributed monitoring agent can be used to monitor the resource usage of each physical node in the power distribution network system in real time, and regularly collect multi-dimensional performance indicator data including CPU utilization, memory usage, disk I / O, network bandwidth, etc. according to the preset sampling frequency, and perform preliminary cleaning and standardization on the collected raw data to obtain a standardized resource usage data set.

[0042] For the normalized resource usage data set, a time window with a fixed size of 10 minutes and a sliding step of 1 minute is used to perform statistical analysis on various performance indicators, and the statistical characteristics of resource usage within the window, such as the mean, peak value, and variance, are calculated. Based on the pre-set multi-level resource threshold system, the calculated statistical characteristic values ​​are cross-compared in multiple dimensions and threshold judgments are made to generate a resource usage status assessment report. Based on the resource occupancy of each node in the resource usage status assessment report, it is determined whether to trigger the resource reallocation mechanism. When it is detected that the resources of a certain node exceed the preset threshold, the container migration assessment process is started.

[0043] Exemplarily, in a power distribution network system, a distributed monitoring agent collects resource usage data of each physical node at a sampling interval of 5 seconds. Taking CPU utilization as an example, the data of 12 sampling points of a node within 1 minute are [75%, 80%, 82%, 78%, 85%, 88%, 90%, 92%, 89%, 87%, 85%, 83%]. These raw data are standardized and converted into standardized values ​​relative to the number of CPU cores of the node. For a node with an 8-core CPU, the standardized data becomes [6, 6.4, 6.56, 6.24, 6.8, 7.04, 7.2, 7.36, 7.12, 6.96, 6.8, 6.64]. A fixed window of 10 minutes and a sliding step of 1 minute are used to perform statistical analysis on the standardized data. In the current window, the mean CPU usage is calculated to be 6.85 cores, the peak is 7.36 cores, and the variance is 0.1296. In the preset multi-level resource threshold system, the warning threshold of CPU usage is 80%, or 6.4 cores, and the danger threshold is 90%, or 7.2 cores. Through multi-dimensional cross-comparison, it is found that the mean and peak values ​​of the CPU usage of this node exceed the warning threshold, and the peak value has reached the danger threshold. Therefore, the generated resource usage status assessment report marks the node as requiring resource reallocation.

[0044] S120. If the resource usage data of the physical node exists and is greater than the preset resource threshold, the overall resource distribution status of the distribution network system is analyzed through a resource evaluation algorithm, the load balancing index of each physical node is calculated, and based on the load balancing index, business priority and service quality requirement information corresponding to the physical node, a container migration task list is generated.

[0045] The container migration task list includes a source node, a target node, a container identifier and a migration time window.

[0046] In an embodiment of the present invention, the resource evaluation algorithm is used to analyze the overall resource distribution status of the power distribution network system, calculate the load balancing index of each physical node, and generate a container migration task list based on the load balancing index, business priority and service quality requirement information corresponding to the physical node, including:

[0047] The overall resource distribution status of the power distribution network system is modeled by a resource evaluation method based on graph theory to obtain a target resource graph, wherein each of the physical nodes is a vertex of the target modeling graph, and the network connection between the physical nodes is an edge of the target modeling graph;

[0048] Acquire resource usage data of each of the physical nodes simultaneously through a parallel data collection method, and acquire business priority and service quality requirement information running on each of the physical nodes;

[0049] According to the resource usage data, a load balancing index of each of the physical nodes is calculated using a weighted average algorithm; according to the load balancing index, the business priority and the service quality requirement information, a container migration decision model is constructed using a non-dominated sorting genetic algorithm, so as to make a decision on each of the physical nodes through the container migration decision model, and determine a plurality of potential migration source nodes and potential migration target nodes in the power distribution network system;

[0050] For the potential migration source node and the potential migration target node, determine the container to be migrated, as well as the source node, target node, container identifier and migration time window of the container to be migrated based on the ant colony optimization algorithm according to the network delay between nodes, bandwidth limitation and migration overhead; add the source node, target node, container identifier and migration time window of each container to be migrated to the task queue to obtain the container migration task list.

[0051] Specifically, a resource evaluation method based on graph theory can be used to model the overall resource distribution of the distribution network system to obtain a target resource graph. In the modeling process, each physical node is regarded as a vertex in the graph, and the network connection between nodes is regarded as an edge.

[0052] Furthermore, through a parallel data collection method, multi-dimensional resource usage data such as CPU utilization, memory usage, disk I / O and network bandwidth of each node are simultaneously obtained, and the business priority and service quality requirement data running on each node are also obtained.

[0053] Based on resource usage data, the weighted average algorithm is used to calculate the load balancing index of each node; based on the node load balancing index, business priority data, and service quality requirement data, a container migration decision model can be constructed through a non-dominated sorting genetic algorithm. If the load balancing index of a node exceeds the threshold, it is marked as a potential migration source. If it is lower than the threshold, it is marked as a potential migration target node; for the marked potential migration source nodes and potential migration target nodes, the ant colony optimization algorithm is used to generate the optimal container migration plan. The ant colony optimization algorithm combines the network delay between nodes, bandwidth limitation, and migration overhead to calculate the source node, target node, and optimal migration time window for each container to be migrated.

[0054] For example, multi-dimensional resource usage data such as CPU utilization, memory usage, disk I / O, network bandwidth, etc. of each node can be obtained, and the business priority and service quality requirement data running on each node can be collected, and all collected data can be stored in a distributed key-value storage system.

[0055] Based on the stored resource usage data, the load balancing index of each node is calculated using a weighted average algorithm, where the CPU weight is 0.4, the memory weight is 0.3, the disk I / O weight is 0.2, and the network bandwidth weight is 0.1. These weight values ​​are determined based on the importance of the resource's impact on system performance. A comprehensive score reflecting the current load status of the node is obtained, and the score result is updated to the distributed key-value storage system.

[0056] Furthermore, the business priority, service quality requirement data and node load balancing index are read from the distributed key-value storage system. Combined with these data, a container migration decision model is constructed through the Non-dominated Sorting Genetic Algorithm II (NSGA-II). The migration source node selection threshold is set to 0.8 and the target node selection threshold is set to 0.5. If the node load balancing index exceeds 0.8, it is marked as a potential migration source node. If it is lower than 0.5, it is marked as a potential migration target node. In the decision-making process, containers with high priority businesses and strict service quality requirements are given higher migration priority.

[0057] For the marked potential migration source nodes and target nodes, the ant colony optimization algorithm is used to generate the optimal container migration plan. Taking into account the network delay between nodes, bandwidth limitation and migration overhead, the source node, target node and optimal migration time window of each container to be migrated are calculated. The migration time window is determined according to the business load cycle and network bandwidth usage, and is usually carried out during the business off-peak period. Finally, a migration task list containing key parameters such as source node ID, target node ID, container ID, migration start time and estimated completion time is output to the task queue manager.

[0058] In the power distribution network system, assuming there are 10 physical nodes, the resource usage data of all nodes are obtained every 5 seconds through the parallel data collection method. Taking node A as an example, the data collected at a certain time is 75% CPU utilization, 60% memory usage, 40% disk I / O usage, and 30% network bandwidth usage. At the same time, it is collected that the service quality requirement of the high-priority business container X running on node A is 99.99% availability. These data are stored in the distributed key-value storage system with the keys NodeA_Resource and ContainerX_QoS. The load balancing index of node A is calculated using the weighted average algorithm, 0.475% + 0.360% + 0.240% + 0.130% = 59%, and the result is updated to the storage system with the key NodeA_Load Index.

[0059] The non-dominated sorting genetic algorithm NSGA-II reads the load balancing index of all nodes and the service quality requirements of the container to perform multi-objective optimization. Since the load balancing index of node A is 59%, which is between the migration source node threshold of 0.8 and the target node threshold of 0.5, migration is not considered for the time being. It is found that the load balancing index of node B is 85%, which exceeds the source node threshold, and it is marked as a potential migration source node. The load balancing index of node C is 45%, which is lower than the target node threshold and is marked as a potential migration target node. Based on this information, the ant colony optimization algorithm considers the network delay from node B to node C to be 10ms, the available bandwidth is 1Gbps, and the container size is 2GB, and calculates the optimal migration plan. It is determined to start the migration at 2:00 am during the off-peak period of business, and the estimated completion time is 2:30. Finally, a migration task list item {source node: B, target node: C, container ID: Y, start time: 2023-07-3002:00:00, estimated completion time: 2023-07-3002:30:00} is generated and added to the task queue manager.

[0060] S130: According to the container migration task list, call the container migration preprocessing module to encapsulate the container data corresponding to the container to be migrated in the migration task list, and determine the container state snapshot information and network connection information of the container to be migrated.

[0061] In the embodiment of the present invention, the container migration task list may be parsed to obtain a container identifier of the container to be migrated, and the container detailed information may be obtained according to the container identifier;

[0062] According to the image name and the version tag, the image of the container to be migrated is incrementally packaged using the commit command of Docker, an incremental layer file is generated by comparing the difference between the base image and the current container file system, and the incremental layer file is merged with the base image into a new image file;

[0063] Use Jinja2 template engine (Jinja2 is a Python-based template engine) to convert the runtime configuration parameters into a standardized configuration file in YAML format;

[0064] Calling a CRIU (Checkpoint / Restore In Userspace) tool to perform a checkpoint recovery function of the container, and generating snapshot information of the container state when the container to be migrated is running; wherein the snapshot information of the container state includes process tree, memory page, and file descriptor information;

[0065] Use the tcpdump tool (dump the traffic on a network, a network packet capture tool) to capture the currently active network connection status of the container to be migrated to obtain network connection information; wherein the network connection information includes the source node IP, the target node IP, the port number and the protocol type network session information.

[0066] The container detailed information includes the image name, version label, runtime configuration parameters, and mounted data volume path.

[0067] Specifically, the JSON format data in the migration task list can be parsed to extract the identification information of the container to be migrated, and the container detailed information, including the image name, version label, runtime configuration parameters, mounted data volume path, etc., can be obtained through the container runtime API, and this information can be saved in a key-value pair data structure for subsequent packaging processing.

[0068] Use Docker's commit command to incrementally encapsulate the image of the container to be migrated. By comparing the differences between the base image and the current container file system, an incremental layer file is generated, and the incremental layer and the base image are merged into a new image file. At the same time, the Jinja2 template engine is used to convert the runtime configuration parameters into a standardized configuration file in YAML format to ensure that the configuration information matches the encapsulated image.

[0069] For the data volume associated with the container, it is encapsulated through the LVM snapshot mechanism, and a snapshot copy of the data volume is created using the write-time copy technology to record the data status at the snapshot creation time. If the data volume is located on the Ceph object storage, the Ceph RBD (snapshot API) is called to perform the remote snapshot operation and associate the snapshot metadata information with the corresponding container configuration. Among them, (RADOS Block Device, RADOS Block Device, referred to as RBD).

[0070] Call the CRIU tool to execute the checkpoint recovery function of the container and generate a memory snapshot of the container's runtime state, including process tree, memory pages, file descriptors and other information. At the same time, use the tcpdump tool to capture the container's currently active network connection status, record network session information such as source IP, target IP, port number, protocol type, etc., save the generated state snapshot file and network connection information to the specified persistent storage path, and add corresponding index information in the container metadata to facilitate subsequent migration and recovery operations.

[0071] For example, in a power distribution network system, it is assumed that a container named power-monitor needs to be migrated. The migration task list contains JSON format data {"container_id":"abc123","source_node":"node1","target_node":"node2"}. After parsing this data, the detailed information of the container is obtained through the DockerAPI. The image name is power-monitor:v1.2, the runtime configuration parameters include the environment variable POLL_INTERVAL=5s, and the mounted data volume path is / var / lib / power-data. This information is saved in the Redis key-value storage with the key migration:abc123. The Dockercommit command is used to incrementally package the container. The base image size is 100MB, the incremental layer is 20MB, and the size of the merged new image is 120MB. The Jinja2 template engine converts the runtime parameters into YAML format, and the data volume uses LVM (Logical Volume Manager) to create a 2GB snapshot named "power-data-snap-20230730". Since the data volume is located locally, there is no need to call CephAPI. The CRIU tool performs a checkpoint operation and generates a snapshot file containing information such as the process tree and memory pages, which is about 500MB in size. At the same time, tcpdump captures that the container currently has three active TCP connections, with source ports 32768, 32769, and 32770, target IP 192.168.1.100, and target port 3306. All generated files are saved to the / var / lib / container-migration / abc123 / directory, and index information is added to the container metadata.

[0072] S140, start the cross-node data transmission module, establish a secure channel between the source node and the target node based on the network connection information, use an incremental transmission algorithm to transmit the container data and the container status snapshot information of the source node to the target node, and verify them through a data verification mechanism during the transmission process.

[0073] In the embodiment of the present invention, an encrypted communication channel can be established through a secure socket layer protocol according to the source node IP and the target node IP in the network connection information, and a session key can be generated using a Diffie-Hellman algorithm (key exchange protocol / algorithm).

[0074] The communication is encrypted using the session key, an rsync algorithm is used to implement block-based incremental transmission, the container data and the container state snapshot information are divided into fixed-size data blocks to be transmitted, and a first SHA-256 checksum of the data blocks to be transmitted is calculated;

[0075] Obtaining the SHA-256 checksum of the data block to be transmitted, compressing the data block to be transmitted using the LZ4 algorithm, and sending the compressed data block to be transmitted to the target node through an encrypted communication channel; the target node receives the compressed data block to be transmitted, decompresses it, and calculates a second SHA-256 checksum;

[0076] If the first SHA-256 checksum and the second SHA-256 checksum are consistent, writing the decompressed data block to be transmitted into the target storage location of the target node;

[0077] After each data block to be transmitted is transmitted, a full SHA-256 check is performed on the container data to determine whether the container data stored in the source node and the target node are consistent.

[0078] The first SHA-256 checksum may be a checksum before the data block to be transmitted is decompressed, and the second SHA-256 checksum may be a checksum after the data block to be transmitted is decompressed.

[0079] Specifically, according to the IP addresses of the source node and the target node, an encrypted communication channel is established through the Secure Sockets Layer protocol, and the Diffie-Hellman algorithm is used to generate a session key for subsequent data transmission encryption and decryption; the communication is encrypted using the session key, and the rsync algorithm is used to implement block-based incremental transmission, the container data is divided into fixed-size data blocks to be transmitted, and the SHA-256 checksum of the data block is calculated; the SHA-256 checksum of the data block is obtained, the data block is compressed using the LZ4 algorithm, and the compressed data block is sent to the target node through the encrypted communication channel; the compressed data block is received, the compressed data block is decompressed, the SHA-256 checksum of the received data is calculated, and compared with the checksum sent by the source node; if the SHA-256 checksum of the received data is consistent with the checksum sent by the source node, the data block is written to the target storage location; after the transmission is completed, the entire container data is fully SHA-256 checked to determine whether the source node and target node data are completely consistent.

[0080] For example, Figure 2 As shown, Figure 2A flow chart of cross-node data transmission data provided in Example 1 of the present invention. According to the source node and target node IP addresses recorded in the container status snapshot information, an encrypted communication channel is established through the Secure Sockets Layer protocol SSL, and the Diffie-Hellman key exchange algorithm is used to generate session keys for encryption and decryption of subsequent data transmission. After the connection is established, the ping tool is used to test the network delay and packet loss rate between nodes. If the network quality is poor, the adaptive transmission control mechanism is started. The rsync algorithm is used to implement block-based incremental transmission, and the container image, configuration file, and data volume are divided into 4KB data blocks. The SHA-256 checksum of each data block is calculated and compared with the existing data of the target node. The changed data blocks that need to be transmitted are identified, and a transmission task queue is generated.

[0081] Sort the data blocks in the transmission task queue by priority, and give priority to the transmission of key configuration files and metadata information. Read the data blocks to be transmitted from the transmission task queue after priority sorting, compress the data using the LZ4 compression algorithm to reduce the amount of transmitted data, and then send the compressed data blocks to the target node through the established secure channel, while recording the transmission progress and network bandwidth usage. Implement the breakpoint resume function, update the transmission status record after each data block is transmitted, and resume the transmission from the last recorded status if the network is interrupted. Dynamically adjust the transmission rate according to the network bandwidth usage to avoid occupying too many network resources.

[0082] After receiving the compressed data block, the target node first decompresses it, then calculates the SHA-256 checksum of the received data, and compares it with the checksum sent by the source node. If the checksum is consistent, the data block is written to the target storage location. If it is inconsistent, the exponential backoff algorithm is used for retransmission. The retry intervals are 1 second, 2 seconds, 4 seconds, and 8 seconds, respectively, and a maximum of 5 retries are performed. After the transmission is completed, the entire container data is fully SHA-256 checked to ensure that the source node and the target node data are completely consistent.

[0083] In the power distribution network system, suppose a container needs to be migrated from the source node with IP 192.168.1.10 to the target node with IP 192.168.1.20. First, an encrypted channel is established through the SSL protocol, and a shared key is generated using the 2048-bit Diffie-Hellman key exchange algorithm. Subsequently, a ping test is performed, and the results show that the average delay is 5ms, the packet loss rate is 0.1%, and the network condition is good. The rsync algorithm starts working and divides the 100MB container data into 25600 4KB data blocks. By comparing the SHA-256 checksum, it is found that there are 2000 data blocks to be transmitted. These data blocks are added to the transmission queue, which contains 10 high-priority configuration file blocks, which are moved to the front of the queue. The LZ4 algorithm compresses the data, with an average compression ratio of 1.8:1, reducing the amount of data to be transmitted to about 4.4MB. After the transmission starts, the initial rate is set to 10MB / s. During the transmission process, the network was briefly interrupted for 3 seconds. The breakpoint resume function was immediately activated and the transmission continued from the last successfully transmitted data block 1756. The dynamic rate adjustment mechanism detected that the bandwidth utilization rate reached 85% and reduced the transmission rate to 8MB / s. At the target node, the decompressed data was SHA-256 verified, and 5 data blocks failed the verification. The system started the exponential backoff algorithm for retransmission. The first retry was performed after 1 second, the second after 2 seconds, and finally all data blocks were successfully transmitted after 4 seconds of the third retry. After the transmission was completed, the entire container data was fully verified by SHA-256. The checksums of the source node and the target node were both a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6, confirming that the data was completely consistent.

[0084] In a preferred embodiment, during the transmission of the container data, the rsync algorithm is used to implement block-based incremental transmission, and a distributed coordination mechanism based on the Raft consensus algorithm can also be started to ensure the consistency of container migration through election, log replication and security check mechanisms.

[0085] Specifically, the container image and business data are transferred in blocks. By setting the parallel transmission parameter -p of rsync to 8, 8 data blocks are transferred at the same time to accelerate data migration. During the migration process, the distributed coordination mechanism based on the Raft consensus algorithm is started to monitor the migration process and synchronize the status. The nodes are divided into three roles: Leader, Follower, and Candidate. The consistency and reliability of container migration are ensured through mechanisms such as election, log replication, and security checks. The Leader node is responsible for receiving the migration instructions and copying them to the Follower node as log entries. When most nodes confirm that they have received the log, the Leader node submits the log and performs the corresponding migration operation.

[0086] S150: After completing the verification, start the calling container reconstruction module on the target node, rebuild the container operating environment according to the container data and the container status snapshot information, and generate a target container.

[0087] The target container may be the same container as the container to be migrated, and the target container may run on the target node to complete the same business processing task as the container to be migrated.

[0088] In an embodiment of the present invention, a container configuration file in the container data may be parsed to obtain key parameters, wherein the key parameters include network settings, storage mount points, and environment variables;

[0089] Creating an instance of the target container consistent with the specification of the container to be migrated through the Docker API according to the key parameters;

[0090] The OverlayFS file system is used to mount the transferred container image file as a read-only layer to the local storage system of the target node, and a new writable layer is created to form the root file system of the target container.

[0091] Use the CRIU tool to read the transmitted container status snapshot information to obtain the memory status, file descriptor and network connection information of the process in the container;

[0092] If the acquisition is successful, the restored process is reloaded into the instance of the target container;

[0093] Calling the cgroups (Control Groups) mechanism to allocate CPU, memory, disk I / O, and network bandwidth resources to the target container, setting the same resource limits as the container to be migrated;

[0094] Collect resource usage indicators of the target container through node_exporter, including CPU usage, memory usage, disk I / O throughput, and network traffic;

[0095] Use the PromQL query language to compare resource usage of the container to be migrated and the target container to determine consistency of resource usage;

[0096] If significant differences occur, the resource reallocation process is triggered.

[0097] Specifically, according to the transmitted container data and status information, the container configuration file is parsed, key parameters such as network settings, storage mount points and environment variables are extracted, and a container instance consistent with the source container specifications is created on the target node through the Docker API.

[0098] Determine the network mode of the source container. If it is bridge mode, create a new network namespace on the target node and assign a new IP address. If it is host mode, use the host network directly. Use the iptables tool to set the same port mapping rules as the source container.

[0099] Using the OverlayFS file system, the transferred container image file is mounted as a read-only layer to the local storage system of the target node, a new writable layer is created, and the root file system of the container is formed. The Dockervolume command is used to create a data volume identical to the source container, and the rsync tool is used to synchronize the source container data volume content to the newly created data volume.

[0100] Use the CRIU tool to read the transferred container state snapshot file and restore the runtime information such as the memory state, file descriptors, and network connections of the process in the container.

[0101] Through the Linux namespaces mechanism, the restored process is reloaded into the newly created container instance to ensure that the process runs in an isolated environment. Use systemd (system and service manager) or supervisor (process management tool) to configure the startup order of service processes in the container and start each service one by one according to the dependency relationship.

[0102] Call the cgroups control group mechanism to allocate resources such as CPU, memory, disk I / O, and network bandwidth to the reconstructed container, and set the same resource limits and priorities as the source container. Start the Prometheus monitoring tool and collect container resource usage indicators through node_exporter, including CPU usage, memory usage, disk I / O throughput, and network traffic.

[0103] Use the PromQL query language to compare the resource usage of the source container and the target container to verify the consistency of resource usage. If there is a significant difference, the resource reallocation process is triggered.

[0104] For example, in a power distribution network system, suppose a container named power-monitor needs to be migrated. After parsing the container configuration file, it is found that the container uses the bridge network mode, the original IP is 172.17.0.5, and port 8080 needs to be exposed. DockerAPI creates a new container on the target node and assigns an IP address of 172.17.0.10. iptab les adds the rule - ADOCKER-ptcp --dport8080-jDNAT --to-destination172.17.0.10:8080. OverlayFS mounts a 100MB image file as a read-only layer and creates a 10MB writable layer. Use docker volume create power-data to create a data volume, and rsync synchronizes 50GB of historical data, which takes about 10 minutes. CRIU reads a 200MB state snapshot file and restores a process tree containing 1 main process and 3 child processes.

[0105] The process is loaded into the new namespace through the setns system call. The systemd configuration file specifies the service startup order database service, cache service, and API service. cgroups sets the CPU limit to 2 cores, the memory limit to 4GB, the disk I / O limit to 100MB / s, and the network bandwidth limit to 1Gbps. Prometheus collects indicators every 15 seconds. The data for the past hour shows that the average CPU utilization rate is 65%, the memory occupancy is 3.2GB, the disk I / O averages 50MB / s, and the network traffic peak is 800Mbps. Through PromQL query abs (source container indicator-target container indicator) / source container indicator>0.1 to detect resource usage differences, the results show that all indicator differences are within 10%, which verifies the consistency of resource usage.

[0106] S160: Pause the container to be migrated on the source node, activate the target container on the target node, update the routing table and load balancing configuration of the power distribution network system, and switch the service traffic of the container to be migrated to the target container.

[0107] In the embodiment of the present invention, pausing the container to be migrated on the source node, activating the target container on the target node, updating the routing table and load balancing configuration of the power distribution network system, and switching the service traffic of the container to be migrated to the target container includes:

[0108] The source node receives a termination instruction carrying an identifier of the source node, where the termination instruction is issued by the Kubernetes system when the container is migrated;

[0109] Sending a SIGTERM signal to the container to be migrated according to the termination instruction, and the container to be migrated performing an application cleanup operation according to the SIGTERM signal;

[0110] Acquire data difference information between the container to be migrated and the target container, wherein the data difference information is obtained by an rsync tool through an incremental synchronization method;

[0111] Using the data difference information to update the HAProxy load balancer configuration, adding the target container to the server pool and setting an initial weight value;

[0112] Determining whether the target container performance index meets a preset condition includes: obtaining a response time and an error rate of the new container, and comparing the response time and the error rate with corresponding values ​​of the original container;

[0113] If the performance index of the target container is higher than that of the container to be migrated, a traffic switching operation is triggered;

[0114] Update the network address translation rules through iptables to redirect the inbound traffic to the target container.

[0115] Specifically, Kubernetes sends an elegant termination instruction to the source node, uses the SIGTERM signal to notify the application in the original container to perform a cleanup operation, and sets a grace period of 30 seconds. At the same time, start a new container on the target node and set the status of the new container to ready. Use the rsync tool to perform the last incremental data synchronization to ensure data consistency. Call the OpenFlow protocol to update the routing table of the distribution network system, and point the routing entry of the original container IP address to the IP address of the new container. Modify the HAProxy load balancer configuration, add the new container to the server pool, and set the initial weight to 20%. Use Lua scripts to implement dynamic weight adjustment, increasing by 20% every 60 seconds until it reaches 100%. Start the Prometheus monitoring system and configure monitoring indicators for the new container, including CPU usage, memory usage, request latency, and error rate. Set Grafana alarm rules to trigger alarms when indicators exceed preset thresholds. Deploy the Goreplay tool, copy production traffic and send it to the new container, and compare the response time and error rate of the new and old containers. If the performance indicators of the new container are better than the old container for 30 seconds, a full switch operation is triggered. Update the network address translation NAT rules through the iptables command to redirect all inbound traffic to the new container. Use the Jaeger distributed tracing system to collect detailed information on the new container processing requests. If an abnormal indicator is detected that exceeds the preset threshold, perform a rollback operation immediately. When rolling back, restore the network connection of the original container and reapply the original routing rules and load balancing configuration. Use Consul key-value storage to save the switching state to ensure consistency during the rollback process. Perform rsync incremental synchronization to synchronize the latest data on the new container back to the original container to ensure data integrity.

[0116] For example, in a power distribution network system, assume that a container named power-monitor needs to be migrated from node A to node B. Kubernetes sends an elegant termination instruction to node A, uses the SIGTERM signal to notify the application in the container to clean up, and sets a grace period of 30 seconds. At the same time, a new container is started on node B and the status is set to ready. rsync performs the last incremental synchronization, transferring 5GB of data in 45 seconds. OpenFlow updates the routing table to point the routing entry of 172.16.0.10 to the new container IP 172.16.0.20. HAProxy configuration is modified, the initial weight of the new container is 20%, and the Lua script increases by 20% every 60 seconds. Prometheus configures the monitoring indicators CPU usage threshold of 80%, memory usage threshold of 2GB, request latency threshold of 200ms, and error rate threshold of 1%. Grafana sets the alarm rule, and the alarm is triggered when the threshold is exceeded for 3 consecutive times. Goreplay copies 100 requests / second of production traffic to the new container. The comparison shows that the average response time of the new container is 50ms, while that of the old container is 65ms; the error rate of the new container is 0.5%, while that of the old container is 0.8%. The performance indicators are better than the old container for 30 seconds, triggering a complete switch. iptables updates the NAT rule -APREROUTING-tnat-ieth0-ptcp--dport80-jDNAT--to172.16.0.20:80. The Jaeger tracking system shows that 99% of the request delays are <100ms. Within 15 minutes after the switch, the Consul key-value storage records the switch status as completed.

[0117] In a preferred embodiment, after the migration is completed, the resource recovery module is triggered to clean up the redundant data on the source node, release the occupied system resources, update the resource pool status information, and record the migration log. Figure 3 As shown, it is a flow chart of resource recycling provided in the first embodiment of the present invention.

[0118] The resource recovery module on the source node is triggered according to the migration completion mark, and the original container instance is deleted through the DockerAPI. After the deletion is completed, the data volume information associated with the container is obtained; the data volume information is used to use LVM to unbind the data volume from the node. If the unbinding is successful, the lvremove command is used to delete the logical volume; after deleting the logical volume, the IP address used by the container is recovered through the IPAM tool, and the DHCP server configuration is updated after the recovery is completed; after updating the configuration, the BleachBit file system cleanup tool is called to scan the directories and files related to the migrated container on the source node. After the scan is completed, the expired data is moved to the specified directory according to the preset retention policy; after moving the data, the node_exporter is used to collect the CPU, memory, disk I / O and network bandwidth resource usage on the source node. After the collection is completed, the node's available resource information is updated through the Kubernetes resource manager; after updating the resource information, the ResourceQuota object is used to verify the released resources. After the verification is completed, it is determined that the resources are correctly recovered to the resource pool.

[0119] Exemplarily, according to the migration completion flag, the resource recovery module on the source node is triggered, the original container instance is deleted through the Docker API, and the binding relationship between the data volume and the node is released by using LVM. The lvremove command is used to delete the logical volume and return the storage space to the volume group. The IP address used by the container is recovered through the IP address management tool IPAM, and the DHCP server configuration is updated. The BleachBit file system cleaning tool is called to scan the directories and files related to the migrated container on the source node, including the / var / log, / tmp, and / var / cache directories. According to the preset retention policy, the expired data is moved to the / var / cleanup / temporary directory. The secure-delete tool is used to securely delete sensitive files to ensure that the data cannot be recovered. Set a cron scheduled task to automatically clean up files older than 7 days in the / var / cleanup / temporary directory at 2 a.m. every day. Use node_exporter to collect resource usage such as CPU, memory, disk I / O, and network bandwidth on the source node, and update the node's available resource information through the Kubernetes resource manager. Use the ResourceQuota object to verify the released resources to ensure that the resources are correctly recycled to the resource pool. Verify changes in node resource indicators by querying the Prometheus API. Start the Fluentd log aggregator to collect all log information generated during the migration process, including container status changes, network configuration updates, and resource usage. Integrate and write this information to the Elast icsearch distributed log storage system. Encrypt logs using X.509 certificates to implement role-based access control. Trigger the Kibana log analyzer to generate migration performance reports and resource utilization trend charts, and identify potential resource leakage issues through anomaly detection algorithms. In the power distribution network system, assume that after a container named power-monitor is migrated, the resource recovery process is started immediately. DockerAPI sends a delete command with a container ID of abc123de4f, which is stopped and deleted within 5 seconds. LVM unbinds the data volume power-data from the node and executes lvremove / dev / vg0 / power-data to release 50GB of storage space. IPAM reclaims the IP address 192.168.1.100 and updates the DHCP server configuration file dhcpd.conf. BleachBit scanned the / var / log, / tmp, and / var / cache directories, identified 2GB of expired logs and 1.5GB of cache files, and moved them to / var / cleanup / temporary. The secure-delete tool overwrote the configuration files containing sensitive information seven times to ensure that the data could not be recovered.The cron task is set to 02***find / var / cleanup / temporary-mtime+7-delete. The node_exporter collection shows that the CPU usage has dropped from 80% to 20%, the available memory has increased by 4GB, and the disk I / O wait time has dropped from 15ms to 5ms. KubernetesResourceQuota verifies the released resources of 2 CPU cores, 8GB memory, and 100GB storage space. Prometheus query API increase(node_cpu_seconds_total{mode='idle'}[1h]) confirms that the CPU idle time has increased. Fluentd aggregates 30MB of log data and writes it to the Elast icsearch index migration-logs-2023.07.30. The logs are protected using the AES-256 encryption algorithm, with a key rotation period of 30 days. Kibana generates a report showing that the migration took 15 minutes and resource utilization increased by 25%. The anomaly detection algorithm finds abnormal network bandwidth usage, indicating potential resource leakage and triggering further investigation.

[0120] The technical solution of the embodiment of the present invention first combines business priorities and service quality requirements to generate the optimal migration plan, and uses incremental transmission and state snapshot technology to achieve efficient migration of containers, thereby improving the efficiency of migration and reducing the complexity of migration; in addition, during the migration process, resource isolation is used to ensure the consistency of the reconstructed container, and a fast fault detection and rollback mechanism is used to ensure business continuity, ensure the reliability of the migration process, and ensure the integrity of the migration data; finally, through dynamic resource scheduling and load balancing, it is ensured that the new container can quickly and accurately build the operating environment, solve the complexity problem of the target node container reconstruction, realize flexible allocation of resources, can quickly adapt to business changes and load fluctuations, reduce system maintenance costs, and reduce manual intervention and errors through automated resource recovery and status updates. In summary, this method realizes the intelligent, automated and dynamic allocation of distribution network system resources, improves the system's reliability and resource utilization, and enhances the system's elasticity and reliability.

[0121] Embodiment 2

[0122] Figure 4 This is a schematic diagram of the structure of a containerized resource dynamic allocation device for a power distribution network system provided in the second embodiment of the present invention. Figure 4 As shown, the device comprises:

[0123] The resource usage judgment module 210 is used to monitor the resource usage status of each physical node in the power distribution network system in real time through the container resource monitoring module, obtain the resource usage data of each physical node, and judge whether there is a physical node whose resource usage data is greater than a preset resource threshold;

[0124] The migration list generation module 220 is used to analyze the overall resource distribution status of the power distribution network system through a resource evaluation algorithm if the resource usage data of the physical node is greater than a preset resource threshold, calculate the load balancing index of each physical node, and generate a container migration task list based on the load balancing index, business priority and service quality requirement information corresponding to the physical node; wherein the container migration task list includes a source node, a target node, a container identifier and a migration time window;

[0125] The container encapsulation module 230 is used to call the container migration preprocessing module according to the container migration task list, encapsulate the container data corresponding to the container to be migrated in the migration task list, and determine the container state snapshot information and network connection information of the container to be migrated;

[0126] The data transmission module 240 is used to start the cross-node data transmission module, establish a secure channel between the source node and the target node based on the network connection information, adopt an incremental transmission algorithm to transmit the container data and the container status snapshot information of the source node to the target node, and verify them through a data verification mechanism during the transmission process;

[0127] The container reconstruction module 250 is used to start the calling container reconstruction module on the target node after completing the verification, rebuild the container operating environment according to the container data and the container state snapshot information, and generate the target container;

[0128] The container migration module 260 is used to suspend the container to be migrated on the source node, activate the target container on the target node, update the routing table and load balancing configuration of the power distribution network system, and switch the service traffic of the container to be migrated to the target container.

[0129] The technical solution of the embodiment of the present invention first combines business priorities and service quality requirements to generate the optimal migration plan, and uses incremental transmission and state snapshot technology to achieve efficient migration of containers, thereby improving the efficiency of migration and reducing the complexity of migration; in addition, during the migration process, resource isolation is used to ensure the consistency of the reconstructed container, and a fast fault detection and rollback mechanism is used to ensure business continuity, ensure the reliability of the migration process, and ensure the integrity of the migration data; finally, through dynamic resource scheduling and load balancing, it is ensured that the new container can quickly and accurately build the operating environment, solve the complexity problem of the target node container reconstruction, realize flexible allocation of resources, can quickly adapt to business changes and load fluctuations, reduce system maintenance costs, and reduce manual intervention and errors through automated resource recovery and status updates. In summary, this method realizes the intelligent, automated and dynamic allocation of distribution network system resources, improves the system's reliability and resource utilization, and enhances the system's elasticity and reliability.

[0130] The containerized resource dynamic allocation device for a distribution network system provided in an embodiment of the present invention can execute the containerized resource dynamic allocation method for a distribution network system provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0131] Embodiment 3

[0132] Figure 5 A schematic diagram of the structure of an electronic device provided for Embodiment 3 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0133] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0134] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0135] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a containerized resource dynamic allocation method for a distribution network system.

[0136] In some embodiments, the method for dynamic allocation of resources for containerized distribution network systems may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for dynamic allocation of resources for containerized distribution network systems described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for dynamic allocation of resources for containerized distribution network systems in any other appropriate manner (e.g., by means of firmware).

[0137] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0139] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0141] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0142] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0144] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A containerized resource dynamic allocation method for a power distribution network system, characterized in that: include: The resource usage status of each physical node in the power distribution network system is monitored in real time through the container resource monitoring module to obtain resource usage data of each physical node, and determine whether there is any physical node whose resource usage data is greater than a preset resource threshold; If the resource usage data of the physical node exists and is greater than the preset resource threshold, the overall resource distribution status of the power distribution network system is analyzed through the resource evaluation algorithm, the load balancing index of each physical node is calculated, and based on the load balancing index, business priority and service quality requirement information corresponding to the physical node, a container migration task list is generated; wherein the container migration task list includes a source node, a target node, a container identifier and a migration time window; According to the container migration task list, calling the container migration preprocessing module, encapsulating the container data corresponding to the container to be migrated in the migration task list, and determining the container state snapshot information and network connection information of the container to be migrated; Starting a cross-node data transmission module, establishing a secure channel between the source node and the target node based on the network connection information, using an incremental transmission algorithm to transmit the container data and the container status snapshot information of the source node to the target node, and verifying them through a data verification mechanism during the transmission process; After completing the verification, the calling container reconstruction module on the target node is started, and the container operating environment is rebuilt according to the container data and the container status snapshot information, and the target container is generated; The container to be migrated on the source node is paused, and the target container on the target node is activated, the routing table and load balancing configuration of the power distribution network system are updated, and the service traffic of the container to be migrated is switched to the target container.

2. The method according to claim 1, characterized in that The resource usage status of each physical node in the power distribution network system is monitored in real time by the container resource monitoring module, resource usage data of each physical node is obtained, and whether the resource usage data of the physical node is greater than a preset resource threshold is determined, including: For the power distribution network system, a distributed monitoring agent is used to monitor the resource usage of each physical node in real time to obtain performance indicator data; wherein the performance indicator data includes CPU utilization, memory usage, disk I / O and network bandwidth; According to the performance indicator data, statistical analysis is performed on each performance indicator to calculate resource usage data, wherein the resource usage data includes a mean value, a peak value, and a variance of resource usage; It is determined whether the resource usage data is greater than a preset resource threshold.

3. The method according to claim 2, characterized in that The resource evaluation algorithm is used to analyze the overall resource distribution status of the power distribution network system, calculate the load balancing index of each physical node, and generate a container migration task list based on the load balancing index, business priority and service quality requirement information corresponding to the physical node, including: The overall resource distribution status of the power distribution network system is modeled by a resource evaluation method based on graph theory to obtain a target resource graph, wherein each of the physical nodes is a vertex of the target modeling graph, and the network connection between the physical nodes is an edge of the target modeling graph; Acquire resource usage data of each of the physical nodes simultaneously through a parallel data collection method, and acquire service priority and service quality requirement information running on each of the physical nodes; Calculating the load balancing index of each of the physical nodes using a weighted average algorithm according to the resource usage data; According to the load balancing index, the service priority and the service quality requirement information, a container migration decision model is constructed by a non-dominated sorting genetic algorithm, so as to make a decision on each of the physical nodes through the container migration decision model, and determine a plurality of potential migration source nodes and potential migration target nodes in the power distribution network system; For the potential migration source node and the potential migration target node, determine the container to be migrated, as well as the source node, target node, container identifier and migration time window of the container to be migrated based on the ant colony optimization algorithm according to the network delay between nodes, bandwidth limitation and migration overhead; The source node, the target node, the container identifier and the migration time window of each container to be migrated are added to the task queue to obtain the container migration task list.

4. The method according to claim 1, characterized in that: The container data includes an image file and runtime configuration parameters. The container migration preprocessing module is called according to the container migration task list to encapsulate the container data corresponding to the container to be migrated in the migration task list, and the container state snapshot information and network connection information of the container to be migrated are determined, including: Parse the container migration task list to obtain the container identifier of the container to be migrated, and obtain container detailed information according to the container identifier, wherein the container detailed information includes the image name, version label and runtime configuration parameters and the mounted data volume path; According to the image name and the version tag, the image of the container to be migrated is incrementally packaged using the commit command of Docker, an incremental layer file is generated by comparing the difference between the base image and the current container file system, and the incremental layer file is merged with the base image into a new image file; Using the Jinja2 template engine to convert the runtime configuration parameters into a standardized configuration file in YAML format; Calling the CRIU tool to execute the checkpoint recovery function of the container, and generating the container state snapshot information when the container to be migrated is running; wherein the container state snapshot information includes the process tree, memory page and file descriptor information; Use the tcpdump tool to capture the currently active network connection status of the container to be migrated to obtain network connection information; wherein the network connection information includes the source node IP, the target node IP, the port number and the protocol type network session information.

5. The method according to claim 1, characterized in that The starting cross-node data transmission module establishes a secure channel between the source node and the target node based on the network connection information, adopts an incremental transmission algorithm to transmit the container data and the container status snapshot information of the source node to the target node, and performs verification through a data verification mechanism during the transmission process, including: According to the source node IP and the target node IP in the network connection information, an encrypted communication channel is established through the secure socket layer protocol, and a session key is generated using the Diffie-Hellman algorithm; The communication is encrypted using the session key, an rsync algorithm is used to implement block-based incremental transmission, the container data and the container state snapshot information are divided into fixed-size data blocks to be transmitted, and a first SHA-256 checksum of the data blocks to be transmitted is calculated; Obtaining the SHA-256 checksum of the data block to be transmitted, compressing the data block to be transmitted using the LZ4 algorithm, and sending the compressed data block to be transmitted to the target node through an encrypted communication channel; The target node receives the compressed data block to be transmitted, decompresses it, and calculates a second SHA-256 checksum; If the first SHA-256 checksum and the second SHA-256 checksum are consistent, writing the decompressed data block to be transmitted into the target storage location of the target node; After each data block to be transmitted is transmitted, a full SHA-256 check is performed on the container data to determine whether the container data stored in the source node and the target node are consistent.

6. The method according to claim 5, characterized in that Also includes: During the transmission of the container data, a distributed coordination mechanism based on the Raft consensus algorithm is started to ensure the consistency of container migration through election, log replication and security check mechanisms.

7. The method according to claim 1, characterized in that After completing the verification, starting the calling container reconstruction module on the target node, rebuilding the container operating environment according to the container data and the container state snapshot information, and generating the target container, includes: Parsing a container configuration file in the container data to obtain key parameters, wherein the key parameters include network settings, storage mount points, and environment variables; Creating an instance of the target container consistent with the specification of the container to be migrated through the Docker API according to the key parameters; The OverlayFS file system is used to mount the transferred container image file as a read-only layer to the local storage system of the target node, and a new writable layer is created to form the root file system of the target container. Use the CRIU tool to read the transmitted container status snapshot information to obtain the memory status, file descriptor and network connection information of the process in the container; If the acquisition is successful, the restored process is reloaded into the instance of the target container; Call the cgroups mechanism to allocate CPU, memory, disk I / O, and network bandwidth resources to the target container, and set the same resource limits as the container to be migrated; Collect resource usage indicators of the target container through node_exporter, including CPU usage, memory usage, disk I / O throughput, and network traffic; Use the PromQL query language to compare resource usage of the container to be migrated and the target container to determine consistency of resource usage; If significant differences occur, the resource reallocation process is triggered.

8. The method according to claim 1, characterized in that The pausing the container to be migrated on the source node, activating the target container on the target node, updating the routing table and load balancing configuration of the power distribution network system, and switching the service traffic of the container to be migrated to the target container includes: The source node receives a termination instruction carrying an identifier of the source node, where the termination instruction is issued by the Kubernetes system when the container is migrated; Sending a SIGTERM signal to the container to be migrated according to the termination instruction, and the container to be migrated performing an application cleanup operation according to the SIGTERM signal; Acquire data difference information between the container to be migrated and the target container, wherein the data difference information is obtained by an rsync tool through an incremental synchronization method; Using the data difference information to update the HAProxy load balancer configuration, adding the target container to the server pool and setting an initial weight value; Determining whether the target container performance index meets a preset condition includes: obtaining a response time and an error rate of the new container, and comparing the response time and the error rate with corresponding values ​​of the original container; If the performance index of the target container is higher than that of the container to be migrated, a traffic switching operation is triggered; Update the network address translation rules through iptables to redirect the inbound traffic to the target container.

9. The method according to claim 1, characterized in that: Also includes: The resource recovery module on the source node is triggered according to the migration completion flag, and the instance of the container to be migrated is deleted through the Docker API. After the deletion is completed, the data volume information associated with the container to be migrated is obtained; Using the data volume information, using LVM to release the binding relationship between the data volume and the source node, and if the release is successful, using the lvremove command to delete the logical volume; After deleting the logical volume, reclaim the IP address used by the container to be migrated through the IPAM tool, and update the DHCP server configuration after the recovery is completed; After the configuration is updated, the BleachBit file system cleaning tool is called to scan the directories and files related to the target container on the source node. After the scan is completed, the expired data is moved to the specified directory according to the preset retention policy; After moving the data, node_exporter is used to collect the CPU, memory, disk I / O, and network bandwidth resource usage on the source node. After the collection is completed, the available resource information of the source node is updated through the Kubernetes resource manager; After updating the resource information, use the ResourceQuota object to verify the released resources. After the verification is completed, it is determined that the resources are correctly recycled to the resource pool.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the containerized dynamic resource allocation method for a distribution network system according to any one of claims 1 to 9.

Images