Preloading method for container migration and distributed file system

Abstract

This invention discloses a preloading method for container migration and a distributed file system. The method includes: pre-storing a list of files required to be loaded when creating and starting a container for each image file in the distributed file system; determining the correlation information between the image files; determining a replica placement node for the target image file based on the correlation information; and enabling the replica placement node to determine a target file list corresponding to the target image file based on the pre-stored list of files required to be loaded when creating and starting a container for each image file, and pre-loading the files required for container creation and startup based on the target file list information. This invention can solve the startup delay problem caused by the superposition of on-demand file loading and resource contention during container migration, and improves the efficiency, success rate, and operational stability of container migration under a distributed file system.

Description

Technical Field

[0001] This invention relates to the field of file system technology, and in particular to a preloading method for container migration and a distributed file system. Background Technology

[0002] Currently, the rail transit industry is undergoing rapid iterative upgrades towards intelligence, networking, and digitalization, leading to an exponential increase in the demand for data collection, processing, and storage on trains. During train operation, massive amounts of data need to be collected, stored, and processed in real time, including data from running gear status monitoring, pantograph-catenary operation monitoring, carriage smoke and fire alarms, and passenger information system video surveillance. The efficient management, reliable storage, and rapid access of this data are crucial for train operation safety, passenger service quality, and overall line operational efficiency, forming a core foundation for intelligent rail transit operation and maintenance. Therefore, lightweight distributed file systems, with their ability to operate efficiently under resource constraints and achieve high-performance data storage and access in the onboard environment, have become the mainstream technical solution for onboard data management and are widely used in rail transit onboard edge cloud scenarios.

[0003] Existing lightweight distributed file systems generally adopt a multi-replica redundancy mechanism, which synchronously stores the image file to multiple on-board server nodes. Through replica redundancy, the system can still provide normal data access and service operation capabilities when some nodes fail, thus maintaining the continuity of train on-board services.

[0004] However, this file system needs to dynamically load application dependency libraries, configuration files, and other initialization elements during container image startup. The image file loading process is time-consuming, which will cause cold start delays in containers, leading to reduced container migration efficiency and decreased system response time to failures. Moreover, since the same image file may exist on multiple adjacent nodes of the system, if the node where the image file copy is placed interferes with the container originally running on that node, it may intensify resource contention within the node, which will also reduce container migration efficiency and affect system performance. Summary of the Invention

[0005] This invention provides a preloading method for container migration to solve the startup delay problem caused by the superposition of on-demand file loading and resource contention during container migration, thereby improving the efficiency and system performance of container migration under a distributed file system. The method is applied to a distributed file system and includes: Pre-store the list of files that need to be loaded when each image file in the distributed file system is created and the container is started; Determine the correlation information between the various image files; Based on the correlation information between each image file, a replica placement node for the target image file is determined; the replica placement node then determines the target file list corresponding to the target image file based on the pre-stored file list required to load when creating and starting the container for each image file, and pre-loads the files required when creating and starting the container based on the target file list information.

[0006] Optionally, a list of files required to be loaded when each image file in the distributed file system is created and the container is started can be stored in advance, including: Based on the access cost and storage cost of each image file under different storage methods, determine the target storage method for the file list of each image file; The file list of each image file is stored according to the target storage method of the file list of each image file.

[0007] Optionally, determine the correlation information between the image files, including: Establish a resource consumption model for each image file; the resource consumption model includes resource consumption information of the image file for different types of resources; Based on the resource consumption model of each image file, determine the interference information between the image files; Based on the interference information between the image files, the correlation information between the image files is determined.

[0008] Optionally, based on the resource consumption model of each image file, interference information between image files is determined, including: Based on the resource consumption information of each image file for the same type of resource, determine the interference information between two image files for the same type of resource. The interference information between the two image files is determined based on the interference information of the same type of resource between the two image files and the preset weights of different types of resources.

[0009] Optionally, based on the interference information between the image files, the correlation information between the image files is determined, including: Based on the preset interference threshold ranges of different correlation levels, the correlation level corresponding to the interference information between each image file is determined.

[0010] Optionally, before determining the node for placing the replica of the target image file based on the correlation information between the image files, the process further includes: Based on the resource consumption model of the target image file, determine the resource information required by the target image file; Based on the resource information required by the target image file, candidate nodes are determined from multiple nodes in the distributed file system; Based on the correlation information between the image files, determine the node where the replica of the target image file will be placed, including: Based on the correlation information between each image file, determine the correlation information between other image files already deployed on the candidate node and the target image file; Based on the correlation information between other image files already deployed on candidate nodes and the target image file, determine the node from the candidate nodes where the copy of the target image file will be placed.

[0011] Optionally, before determining the node from which to place the replica of the target image file based on the correlation information between other image files already deployed on candidate nodes and the target image file, the process further includes: Obtain node resource usage information and node health information of candidate nodes; Based on the correlation information between other image files already deployed on candidate nodes and the target image file, determine the nodes from the candidate nodes where the replicas of the target image file will be placed, including: Based on the correlation information between other image files already deployed on candidate nodes and the target image file, node resource usage information, and node health information, determine the node from the candidate nodes to place the replica of the target image file.

[0012] This invention also provides a distributed file system to address the startup delay caused by the combined effects of on-demand file loading and resource contention during container migration, thereby improving the efficiency and system performance of container migration under the distributed file system. The distributed file system includes: The pre-storage module is used to pre-store the list of files that need to be loaded when each image file in the distributed file system is created and the container is started; The correlation determination module is used to determine the correlation information between the image files; The replica placement node determination module is used to determine the replica placement node of the target image file based on the correlation information between each image file; so that the replica placement node determines the target file list corresponding to the target image file according to the pre-stored file list required to load when creating and starting the container for each image file, and pre-loads the files required when creating and starting the container according to the target file list information.

[0013] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described container migration preloading method.

[0014] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described preloading method for container migration.

[0015] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described container migration preloading method.

[0016] In this embodiment of the invention, a list of files required to be loaded when creating and starting a container for each image file in the distributed file system is pre-stored; correlation information between image files is determined; and a replica placement node for the target image file is determined based on the correlation information between image files. This allows the replica placement node to determine the target file list corresponding to the target image file based on the pre-stored list of files required to be loaded when creating and starting a container for each image file, and to pre-load the files required for container creation and startup based on the target file list information. In this way, during container migration, the replica placement node can determine the target file list corresponding to the target image file based on the pre-stored list of files required to be loaded when creating and starting a container for each image file, and pre-load the files required for container creation and startup based on the target file list information. This solves the startup delay problem caused by the superposition of on-demand file loading and resource contention during container migration, improving the efficiency, success rate, and operational stability of container migration under the distributed file system. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] In the attached diagram: Figure 1 This is a flowchart of a preloading method for container migration provided in an embodiment of the present invention; Figure 2 This is a flowchart of a method for determining correlation information between image files provided in an embodiment of the present invention; Figure 3 This is a structural diagram of a distributed file system provided in an embodiment of the present invention; Figure 4 This is a structural diagram of a correlation determination module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0020] In the description of this specification, the terms "comprising," "including," "having," and "containing" are open-ended terms, meaning that they include but are not limited to. The terms "an embodiment," "a specific embodiment," "some embodiments," and "for example," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. The order of steps involved in the various embodiments is used to illustrate the implementation of this application, and the order of steps is not limited and can be adjusted appropriately as needed.

[0021] Research has revealed that existing lightweight distributed file systems experience significant delays in loading container images during startup, leading to cold start latency issues. In rail transit vehicle-mounted edge cloud applications, file systems have even higher requirements for real-time performance and availability, further amplifying the negative impacts of cold start: firstly, delayed container startup reduces container migration efficiency, prolongs recovery time for failed nodes, and diminishes the system's ability to respond quickly to unexpected events; secondly, cold start results in low system resource utilization; and thirdly, cold start causes slow response times for onboard applications, reducing overall service quality and user experience.

[0022] Furthermore, in order to improve system redundancy and availability, the same image file may exist on multiple adjacent nodes of the system. If the node where the image file copy is placed interferes with the container that was originally running on that node, it may lead to increased resource competition within the node, thereby reducing the migration speed of the container and affecting system performance.

[0023] In response to the above research, this invention provides a preloading scheme for container migration to solve the technical problems existing in the current lightweight distributed file system during container image startup.

[0024] like Figure 1 The diagram shows a flowchart of a preloading method for container migration provided by an embodiment of the present invention. This method is applied to a distributed file system and may include: Step 101: Pre-store the list of files that need to be loaded when each image file in the distributed file system is created and the container is started; Step 102: Determine the correlation information between the image files; Step 103: Determine the replica placement node of the target image file based on the correlation information between each image file; so that the replica placement node determines the target file list corresponding to the target image file according to the pre-stored file list required to load when creating and starting the container for each image file, and pre-loads the files required when creating and starting the container according to the target file list information.

[0025] In this embodiment of the invention, the distributed file system pre-stores the list of files required to be loaded when creating and starting a container for each image file; determines the correlation information between each image file; and determines the replica placement node for the target image file based on the correlation information between each image file. In this way, the replica placement node can determine the target file list corresponding to the target image file during container migration based on the pre-stored list of files required to be loaded when creating and starting a container for each image file, and pre-load the files required when creating and starting the container based on the target file list information. This solves the startup delay problem caused by the superposition of on-demand file loading and resource contention during container migration, and improves the efficiency, success rate, and operational stability of container migration under the distributed file system.

[0026] The following is about Figure 1 The preloading method for container migration is described in detail.

[0027] In step 101 above, firstly, it is necessary to obtain the list of files that need to be loaded when each image file is created and the container is started in the distributed file system. Then, the obtained file list of each image file is stored in the distributed file system.

[0028] The file list can refer to a collection of all file objects that are actually accessed and read by a given image file during the creation and startup of the container; this collection can contain the following information: File path: The absolute path to the file to be loaded or a path relative to the container root directory; File size: Indicates the size of the file, used to allocate storage space during preloading; File type: Indicates the type of file (e.g., regular file, directory, link). File permissions: Defines the access permissions for files; File checksum: Used to verify the integrity of a file during the loading process; Dependency: Optional field, indicating whether the current file depends on other files, and a list of dependent files.

[0029] In practice, a dynamic analysis method can be used to obtain the file list of each image file. That is, a container is created based on the image file in the real running environment, and the file system calls initiated by the container process at startup are monitored, including operations such as open(), read(), mmap(), etc., and the corresponding file paths and access behaviors are captured. The monitoring process can be implemented using tools such as Strace or Ltrace. To reduce the impact on system performance, sampling monitoring (sampling once at regular intervals) or asynchronous monitoring (placing the monitoring task in a background thread for execution) is supported.

[0030] This dynamic analysis method can capture dynamically loaded files that static scans cannot cover, thus accurately obtaining the list of files actually loaded when each image file is created and the container is started.

[0031] To ensure the accuracy and timeliness of the file list, the distributed file system can periodically or when triggered by specific events (e.g., image file updates) re-execute dynamic analysis and generate a new file list to replace it. For example, file list updates can be triggered by a scheduled task that periodically checks if there are new image files to analyze or if there are old file lists to update. When an update is needed, the system re-executes the dynamic analysis process to generate a new file list. To ensure system availability, the update process can employ the following strategies: version control, assigning a version number to each file list, with the system maintaining multiple versions of the list simultaneously; gradual rollout, applying the new file list to different container instances and monitoring its performance; and a rollback mechanism, allowing for rapid rollback if the new file list causes performance degradation or errors.

[0032] It should be noted that, in this embodiment of the invention, the list of files to be loaded can be determined by dynamically analyzing system call behavior during the image startup process; alternatively, it can be determined by statically parsing the image configuration file, startup script, and application code; furthermore, in this embodiment of the invention, the list of files to be loaded can also be determined by combining dynamic analysis and static parsing results for cross-validation and supplementation. No specific limitations are made here.

[0033] In practice, file storage or metadata storage can be used to persistently store the list of files that need to be loaded when each image file is created and the container is started in a distributed file system.

[0034] In one embodiment, in step 101 above, the storage method for the file list can be determined and stored in the following manner: Based on the access cost and storage cost of each image file under different storage methods, determine the target storage method for the file list of each image file; The file list of each image file is stored according to the target storage method of the file list of each image file.

[0035] In practical implementation, when the storage method is file storage, the access cost is the average disk read / write (I / O) time required to read the file list, and the storage cost is the disk space overhead required to store the file list; when the storage method is metadata storage, the access cost is the average time required to read metadata from the metadata management module of the distributed file system, and the storage cost is the memory overhead required to expand the metadata management module to support the storage of the file list.

[0036] The total cost of each image file under different storage methods can be calculated using the following formula (1). : Formula (1) in, For access costs, For storage costs, As a weighting factor for access cost, 1- This is a weighting factor for storage costs. The weighting factor is used for balancing. and The relationships between them can be pre-configured based on different needs. For example, if the distributed file system prioritizes access speed and needs to maximize container startup speed, it can be configured with higher speeds. If the distributed file system has limited storage resources and needs to reduce storage overhead, a lower value can be set. value.

[0037] In practice, the total cost of each image file under different storage methods can be considered. Choose the storage method with the lowest total cost as the target storage method.

[0038] If the target storage method is file storage, the file list is converted into files and stored in a specified directory of the distributed file system. The file name can use the image ID and add a specified suffix (e.g., .preload). In addition, the distributed file system also needs to maintain the mapping relationship between the image ID and the file list file.

[0039] If the target storage method is metadata storage, the file list can be stored as metadata for the image file in the metadata management module of the distributed file system. This storage method requires the distributed file system to be able to extend its metadata management module to support storing the file list; in addition, the distributed file system also needs to provide corresponding APIs to manage the file list, including creating, reading, updating, and deleting the list.

[0040] In this way, storing the file list based on the above method can achieve fine-grained adaptation and dynamic optimization of the file list storage strategy without changing the original file system architecture. This effectively alleviates the I / O bottleneck or memory waste caused by a single storage method and improves the overall efficiency and resource utilization rationality of the preloading stage during container migration.

[0041] In step 102 above, the correlation information between each image file is determined by analyzing the correlation between each image file in the distributed file system.

[0042] In practical implementation, correlation information can refer to a qualitative or quantitative description of the degree of performance interference that may occur when two image files run together on the same node due to sharing underlying physical resources. The correlation information between image files can be determined based on the resource consumption information of each image file during node operation.

[0043] In one embodiment, such as Figure 2 The diagram shown is a flowchart of a method for determining the correlation information between image files according to an embodiment of the present invention, which may specifically include: Step 201: Establish a resource consumption model for each image file; the resource consumption model includes resource consumption information of the image file for different types of resources; Step 202: Determine the interference information between each image file based on the resource consumption model of each image file; Step 203: Determine the correlation information between each image file based on the interference information between each image file.

[0044] In specific implementation, in step 201 above, to analyze the correlation between image files, a resource consumption model for each image file can be established. The resource consumption model can be a quantitative description structure used to characterize the image file's consumption of computing, storage, and communication resources during the creation and startup of the container. The resource consumption model includes information on the image file's resource consumption of different types of resources (e.g., CPU, memory, disk I / O, network bandwidth).

[0045] Specifically, the resource consumption model of an image file can be represented as a vector, where each element of the vector represents the amount of a type of resource consumed by the image file (i.e., resource consumption information). The vector (resource consumption model) can be represented by formula (2): Formula (2) in, For image files Resource consumption model; For mirror CPU consumption For image files Memory consumption For image files Disk I / O consumption, For image files Network bandwidth consumption.

[0046] In practice, resource consumption information can refer to the expected or measured consumption of such resources by the image file during the creation and startup of the container. Alternatively, resource consumption information can be obtained through static analysis and dynamic monitoring. Static analysis methods include analyzing the image's configuration files, startup scripts, application code, etc., to extract resource requirement information, such as analyzing application memory usage and peak CPU utilization, and using the analysis results as the initial values ​​for the resource consumption model. Dynamic monitoring methods use performance analysis tools (e.g., top, vmstat, iostat) to monitor the container's resource usage, record resource consumption data under different loads, and use statistical methods (e.g., average, maximum, variance) or machine learning methods (e.g., regression analysis) to build a resource consumption model. The results can be used to correct the static analysis results and improve the accuracy of the resource consumption model.

[0047] In specific implementation, in step 202, interference information between image files can be determined based on the resource consumption information of the same type of resource between the image files.

[0048] In one embodiment, step 202 above may specifically include: Based on the resource consumption information of each image file for the same type of resource, determine the interference information between two image files for the same type of resource. The interference information between the two image files is determined based on the interference information of the same type of resource between the two image files and the preset weights of different types of resources.

[0049] The interference information between two image files on the same type of resource can be a quantitative measure of the likelihood of performance degradation due to competition for the same type of resource when the two image files are running concurrently on the same node. Therefore, the individual interference information of the two image files on that type of resource can be determined by the ratio of the sum of their consumption of that type of resource to the total capacity of that type of resource on the node.

[0050] In practice, a resource contention function can be pre-established to define the interference information between the image files, thereby quantifying the degree of mutual influence between the two image files when they run on the same node. The resource contention function is shown in formula (3): Formula (3) in, For image files and image file The degree of competition among them for the same type of resources (i.e., interference information). and Image files and image file The consumption of the same type of resource, This represents the total resource capacity of the node. The function means that if the sum of the resource consumption of two image files exceeds the total resource capacity of the node, the contention level is the proportion of the excess; otherwise, it is 0. The value range is [0,1]. For example, 0 means no interference and 1 means that the resources are completely saturated, resulting in serious competition.

[0051] Based on the above formula (3), for each type of resource, the interference information between the two mirror files for that type of resource is calculated. Finally, a weighted average method is used to combine the interference information of different types of resources between the two mirror files to obtain the mirror file. The interference information between the image file j and the mirror file j can be weighted and averaged using the following formula (4): Formula (4) in, For image files and image file Interference information between them For image files and image file Interference information affecting the CPU; For image files and image file Information that interferes with memory; For image files and image file Information on interference with disk I / O; For image files and image file Information regarding interference with network bandwidth.

[0052] , , and The weights for CPU, memory, disk I / O, and network bandwidth are respectively, and: Specifically, the weights of CPU, memory, disk I / O, and network bandwidth mentioned above can be adjusted according to the actual situation. For example, if the system is more sensitive to disk I / O performance, the weights can be increased appropriately. The value of .

[0053] In specific implementation, in step 203 above, determining the correlation information between each image file based on the interference information between each image file can be achieved by directly using the interference information between each image file calculated in the above steps as the correlation information between each image file. This serves as correlation information between the various image files. It can be an interference metric value between 0 and 1.

[0054] Alternatively, the correlation level of interference information between each image file can be determined based on preset interference thresholds for different correlation levels. Specifically, the correlation levels calculated between the images can be determined based on preset interference thresholds for different correlation levels. The images are divided into different correlation levels, and the correlation level corresponding to the interference information between the images is used as the correlation information between the images.

[0055] For example, when When, it is defined as high correlation; when When defined as moderate correlation; when Defined as low correlation. Among them, and This is a preset interference threshold, which can be adjusted according to actual conditions. For example, when the resources of the distributed file system are scarce, it can be reduced. The value is adjusted to reduce the probability of highly correlated image files being placed on the same node.

[0056] In one embodiment, the correlation information between the image files can be represented by a correlation matrix, as shown in the following formula (5): Formula (5) in, Represents the correlation matrix. Indicates image file and image file The correlation information and correlation level between them When considering the relevance level, it can be "High" (i.e., high relevance), "Mid" (i.e., medium relevance), or "Low" (i.e., low relevance). In the case of interference information, the calculated .

[0057] In this way, by establishing a resource consumption model for each image file, the resource consumption information of the image file for different types of resources is determined. Based on the resource consumption model, the interference information between any two images is calculated, reflecting their potential competitive relationship at the multi-dimensional resource level. Then, based on the interference information, the correlation information is determined. This process realizes the analysis from the original image behavior to the node-level deployment strategy, so that the placement of replicas no longer depends on empirical rules or static topology, but is based on measurable, verifiable, and iterative resource behavior modeling, thereby improving the success rate of container migration and the overall resource utilization efficiency of the system, and also improving the efficiency of container migration.

[0058] Moreover, by acquiring interference information of two image files on various resources and combining it with preset weights for weighted fusion, the embodiments of the present invention achieve a unified quantitative description of multi-dimensional resource interference. On this basis, it avoids the bias caused by a single resource dimension dominating the interference assessment and improves the ability of interference information to represent the interaction impact of the real operating environment. This provides a reliable data foundation for subsequent classification of correlation levels based on interference thresholds and optimization of replica placement strategies.

[0059] In addition, embodiments of the present invention map interference information to different correlation levels to form structured correlation information; thereby, it can shield interference from minor differences, avoid blind full pre-storage, significantly reduce storage redundancy, improve processing efficiency, and shorten container cold start latency.

[0060] In practice, in step 103 above, the node for placing the copy of the target image file can be determined based on the correlation information between the image files.

[0061] The target image file can refer to the image file on which a new copy needs to be placed; the copy placement node can refer to the target physical or virtual node in the distributed file system used to carry the new copy of the image.

[0062] For example, based on the correlation information between the image files, the node containing the image file that has the weakest correlation with the target image file can be determined as the node where the copy of the target image file is placed.

[0063] In one embodiment, before step 103 above, the resource information required by the target image file can be determined according to the resource consumption model of the target image file; and candidate nodes can be determined from multiple nodes of the distributed file system according to the resource information required by the target image file. Step 103 above can also be achieved in the following way: Based on the correlation information between each image file, determine the correlation information between other image files already deployed on the candidate node and the target image file; Based on the correlation information between other image files already deployed on candidate nodes and the target image file, determine the node from the candidate nodes where the copy of the target image file will be placed.

[0064] In practice, the resource consumption model of the target image file can be used to estimate the amount of various types of resources required to place the target image file (i.e., the resource information required by the target image file). The resource consumption model of the target image file can be referred to the description in the above embodiments, and will not be elaborated here.

[0065] Then, select nodes from the distributed file system that have sufficient resources of various types to place the target image file as candidate nodes. For example, exclude nodes whose remaining resources are insufficient to run the target image file and / or nodes whose current load (the number of image files already run) exceeds a preset load.

[0066] Next, based on the correlation information between the various image files, the correlation information between other image files already deployed on the candidate nodes and the target image file is determined. The candidate node containing the image file with the weakest correlation to the target image file can be selected as the node for placing a replica of the target image file.

[0067] In this way, by selecting candidate nodes based on the resources required by the target image file, and then determining the replica placement node of the target image file based on the correlation information between other image files already deployed on the candidate nodes and the target image file, we can ensure the resource feasibility of the replica placement node, and avoid the high interference between other image files already deployed on the replica placement node and the target image file. This effectively alleviates the startup delay and performance jitter caused by node resource contention and image behavior coupling during container migration, and improves the service reliability and scheduling intelligence of the distributed file system in high-density container deployment scenarios.

[0068] In practical implementation, to improve processing efficiency, relevance filtering conditions can be preset. For example, only other image files with high or low relevance can be filtered, or only other image files with relevance less than a preset threshold can be filtered. After obtaining the relevance information between other image files deployed on candidate nodes and the target image file, candidate nodes whose relevance information meets the preset relevance filtering conditions are further identified. These are the candidate nodes after initial relevance filtering (i.e., excluding candidate nodes whose relevance does not meet the relevance filtering conditions). Based on the relevance information between other image files deployed on the candidate nodes after initial relevance filtering and the target image file, the replica placement node for the target image file is determined from these candidate nodes. This reduces data processing volume and improves the efficiency of determining the replica placement node while ensuring accuracy.

[0069] In one embodiment, to further improve the accuracy of determining the replica placement node, it may also include: Obtain node resource usage information and node health information of candidate nodes; Based on the correlation information between other image files already deployed on candidate nodes and the target image file, the node for placing the replica of the target image file is determined from the candidate nodes, which may include: Based on the correlation information between other image files already deployed on candidate nodes and the target image file, node resource usage information, and node health information, determine the node from the candidate nodes to place the replica of the target image file.

[0070] In practice, each candidate node is comprehensively scored from multiple dimensions, including the correlation information between other image files already deployed on the candidate node and the target image file, the node resource usage information of the candidate node, and the node health information, so as to determine the node where the replica of the target image file is placed.

[0071] Specifically, for example, a candidate node may have multiple other deployed image files. If a candidate node has many image files that are highly correlated with the target image file, its score is reduced; conversely, if it has few, its score is increased. The correlation score is determined based on the correlation information between the deployed image files and the target image file. It can be calculated using formula (6): Formula (6) in, Other image files on the candidate node that have low correlation with the target image file The weight, Other image files on the candidate node that are highly correlated with the target image file The weight, It is a regulating factor. , , All settings can be configured based on actual needs.

[0072] Specifically, node resource usage information can reflect the current resource utilization status of candidate nodes, including CPU utilization, memory utilization, disk I / O throughput, network bandwidth utilization, and storage space utilization. A resource balance score is calculated based on this information, and this score is used to evaluate whether the node's resource utilization is reasonable after placing the target image file on it. It can be calculated using formula (7): Formula (7) in, This indicates the utilization rate of the resource with the highest utilization rate among all resources (CPU, memory, disk I / O, network bandwidth) on the candidate node, which reflects the utilization of the most strained resource on the node. To measure the degree of balance in storage space, and: in, Storage space utilization rate of candidate nodes. This represents the average storage space utilization rate across all candidate nodes. If the storage space utilization rate of a candidate node is close to the average storage space utilization rate, A value close to 1 indicates that storage space is being used reasonably.

[0073] Specifically, node health information reflects the operational stability and service reliability of candidate nodes, and may include node error rate, average response latency, number of lost heartbeats, hardware fault alarm status, and service availability. A node health score is calculated based on this information, and this score is used to assess the health status of candidate nodes. This score can be obtained by quantifying and weighting the health information of each node.

[0074] Based on the above correlation scores Resource balance score Node health score The comprehensive score of the candidate node is calculated using formula (8). : Formula (8) in, , and This is a weighting factor that can be set based on actual needs.

[0075] Finally, after calculating the overall score for each candidate node, the candidate node with the highest score is selected to place the node for the copy of the target image file.

[0076] In this way, by using the above method, the correlation information between other image files deployed on the candidate node and the target image file, the node resource usage information, and the node health information are collaboratively modeled and jointly scored. This makes the replica placement decision not only consider static resource matching and interference relationships between images, but also incorporates dynamic operational quality dimensions, thereby achieving an objective assessment of the node's comprehensive carrying capacity. As a result, during container migration, it can effectively avoid replica startup failures or performance degradation caused by hidden node faults, sudden loads, or local resource overloads, and improve the high availability and robustness of the distributed file system in supporting container services.

[0077] The container migration preloading method provided in this invention pre-stores a list of files required to be loaded when creating and starting a container for each image file in the distributed file system. During subsequent container migration, files can be pre-loaded according to the file list, thereby effectively reducing container startup latency. Moreover, by analyzing the correlation between image files, the method avoids placing image files with high resource contention on the same node, which can reduce resource contention within the node and improve container migration speed.

[0078] In addition, the preloading method for container migration improves the speed of container migration, enabling the rapid migration of critical application containers from risky server nodes to other normally functioning server nodes in practical applications. This reduces service interruption time, ensures high availability of in-vehicle applications, and can efficiently cope with situations such as load imbalance, server node failure, and software updates and maintenance.

[0079] This invention also provides a distributed file system, as described in the following embodiments. Since the principle behind this distributed file system's solution is similar to the preloading method for container migration described above, the implementation of this distributed file system can refer to the implementation of the preloading method for container migration; details that are repeated will not be repeated.

[0080] like Figure 3 The diagram shown is a structural diagram of a distributed file system provided in an embodiment of the present invention. The distributed file system may include: The pre-storage module is used to pre-store the list of files that need to be loaded when each image file in the distributed file system is created and the container is started; The correlation determination module is used to determine the correlation information between the image files; The replica placement node determination module is used to determine the replica placement node of the target image file based on the correlation information between each image file. This enables the replica placement node to determine the target file list corresponding to the target image file based on the pre-stored file list required to create and start the container for each image file, and to pre-load the files required for container creation and startup based on the target file list information.

[0081] In specific implementation, such as Figure 3 As shown, the distributed file system comprises multiple nodes. A pre-storage module pre-stores the list of files required to create and start a container for each image file in the distributed file system. A correlation determination module determines the correlation information between the image files. A replica placement node determination module determines the node to place the replica of the target image file based on the correlation information determined by the correlation determination module. For example, a replica of image A is placed on node 2, and a replica of image B is placed on node 3. If node 1 crashes, since the replica of image A located on node 1 is placed on node 2, and node 2 has pre-loaded the files required for container creation and startup based on the file list of image A from the pre-storage module, the replica of image A on node 2 can immediately create and start the container based on the pre-loaded files required for container creation and startup.

[0082] In one embodiment, the pre-storage module can also be used for: Based on the access cost and storage cost of each image file under different storage methods, determine the target storage method for the file list of each image file; The file list of each image file is stored according to the target storage method of the file list of each image file.

[0083] In one embodiment, such as Figure 4 As shown, the correlation determination module may include: The resource model building unit is used to build a resource consumption model for each image file; the resource consumption model includes resource consumption information of the image file for different types of resources; The interference determination unit is used to determine the interference information between each image file based on the resource consumption model of each image file. The correlation determination unit is used to determine the correlation information between each image file based on the interference information between each image file.

[0084] In one embodiment, the interference determination unit can also be used for: Based on the resource consumption information of each image file for the same type of resource, determine the interference information between two image files for the same type of resource. The interference information between the two image files is determined based on the interference information of the same type of resource between the two image files and the preset weights of different types of resources.

[0085] In one embodiment, the correlation determination unit can also be used for: Based on the preset interference threshold ranges of different correlation levels, the correlation level corresponding to the interference information between each image file is determined.

[0086] In one embodiment, a first information acquisition module may be further included, configured to determine the replica placement node of the target image file before the replica placement node determination module determines the replica placement node of the target image file based on the correlation information between the image files: Based on the resource consumption model of the target image file, determine the resource information required by the target image file; Based on the resource information required by the target image file, candidate nodes are determined from multiple nodes in the distributed file system; The module for determining replica placement nodes can also be used for: Based on the correlation information between each image file, determine the correlation information between other image files already deployed on the candidate node and the target image file; Based on the correlation information between other image files already deployed on candidate nodes and the target image file, determine the node from the candidate nodes where the copy of the target image file will be placed.

[0087] In one embodiment, a second information acquisition module is further included, configured to determine the replica placement node of the target image file from the candidate nodes based on the correlation information between other image files deployed on the candidate nodes and the target image file: Obtain node resource usage information and node health information of candidate nodes; The module for determining replica placement nodes can also be used for: Based on the correlation information between other image files already deployed on candidate nodes and the target image file, node resource usage information, and node health information, determine the node from the candidate nodes to place the replica of the target image file.

[0088] This invention also provides a computer device. Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 500 includes a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 530, it implements the above-mentioned container migration preloading method.

[0089] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described preloading method for container migration.

[0090] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described preloading method for container migration.

[0091] In this embodiment of the invention, a list of files required to be loaded when creating and starting a container for each image file in the distributed file system is pre-stored; correlation information between image files is determined; and a replica placement node for the target image file is determined based on the correlation information between image files. This allows the replica placement node to determine the target file list corresponding to the target image file based on the pre-stored list of files required to be loaded when creating and starting a container for each image file, and to pre-load the files required for container creation and startup based on the target file list information. In this way, during container migration, the replica placement node can determine the target file list corresponding to the target image file based on the pre-stored list of files required to be loaded when creating and starting a container for each image file, and pre-load the files required for container creation and startup based on the target file list information. This solves the startup delay problem caused by the superposition of on-demand file loading and resource contention during container migration, improving the efficiency, success rate, and operational stability of container migration under the distributed file system.

[0092] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0096] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A preloading method for container migration, characterized by, Applied to distributed file systems, including: Pre-store the list of files that need to be loaded when each image file in the distributed file system is created and the container is started; Determine the correlation information between the various image files; Based on the correlation information between each image file, a replica placement node for the target image file is determined; the replica placement node then determines the target file list corresponding to the target image file based on the pre-stored file list required to load when creating and starting the container for each image file, and pre-loads the files required when creating and starting the container based on the target file list information.

2. The method of claim 1, wherein, The system pre-stores a list of files required to be loaded when creating and starting a container from each image file in the distributed file system, including: Based on the access cost and storage cost of each image file under different storage methods, determine the target storage method for the file list of each image file; The file list of each image file is stored according to the target storage method of the file list of each image file.

3. The method of claim 1, wherein, Determine the correlation information between the various image files, including: Establish a resource consumption model for each image file; the resource consumption model includes resource consumption information of the image file for different types of resources; Based on the resource consumption model of each image file, determine the interference information between the image files; Based on the interference information between the image files, the correlation information between the image files is determined.

4. The method of claim 3, wherein, Based on the resource consumption model of each image file, the interference information between image files is determined, including: Based on the resource consumption information of each image file for the same type of resource, determine the interference information between two image files for the same type of resource. The interference information between the two image files is determined based on the interference information of the same type of resource between the two image files and the preset weights of different types of resources.

5. The method of claim 4, wherein, Based on the interference information between the image files, the correlation information between the image files is determined, including: Based on the preset interference threshold ranges of different correlation levels, the correlation level corresponding to the interference information between each image file is determined.

6. The method of claim 1, wherein, Before determining the node for placing the replica of the target image file based on the correlation information between the image files, the process also includes: Based on the resource consumption model of the target image file, determine the resource information required by the target image file; Based on the resource information required by the target image file, candidate nodes are determined from multiple nodes in the distributed file system; Based on the correlation information between the image files, determine the node where the replica of the target image file will be placed, including: Based on the correlation information between each image file, determine the correlation information between other image files already deployed on the candidate node and the target image file; Based on the correlation information between other image files already deployed on candidate nodes and the target image file, determine the node from the candidate nodes where the copy of the target image file will be placed.

7. The method of claim 6, wherein, Before determining the node to place the replica of the target image file from the candidate nodes, based on the correlation information between other image files already deployed on the candidate nodes and the target image file, the process also includes: Obtain node resource usage information and node health information of candidate nodes; Based on the correlation information between other image files already deployed on candidate nodes and the target image file, determine the nodes from the candidate nodes where the replicas of the target image file will be placed, including: Based on the correlation information between other image files already deployed on candidate nodes and the target image file, node resource usage information, and node health information, determine the node from the candidate nodes to place the replica of the target image file.

8. A distributed file system, characterized by include: The pre-storage module is used to pre-store the list of files that need to be loaded when each image file in the distributed file system is created and the container is started; The correlation determination module is used to determine the correlation information between the image files; The replica placement node determination module is used to determine the replica placement node of the target image file based on the correlation information between each image file; so that the replica placement node determines the target file list corresponding to the target image file according to the pre-stored file list required to load when creating and starting the container for each image file, and pre-loads the files required when creating and starting the container according to the target file list information.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the preloading method for container migration as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the preloading method for container migration as described in any one of claims 1 to 7.

Images