File processing method, system, cluster, apparatus, device, and program product

Abstract

The application provides a file processing method, system, cluster, device, equipment and program product. The method comprises the following steps: determining a target storage area where a mirror file is located, the mirror file being a file obtained by packing at least one associated file dependent by a target application during running; and mounting the mirror file to a container used for running the target application according to the target storage area. The efficiency of starting the application in the container by the dependent mirror file is improved.

Description

Technical Field

[0001] This application relates to the field of container services, and more particularly to a file processing method, system, cluster, device, equipment, and program product. Background Technology

[0002] Multiple containers can run in a cluster. Applications can run in containers, and each application depends on at least one application file and at least one associated file.

[0003] In related technologies, at least one associated file (e.g., a model file) and at least one application file are typically packaged to generate an image file. This image file can then be loaded, and the application can be started and run within a container based on it. However, in this approach, when at least one associated file or at least one application file needs to be updated, a new image file must be generated, loaded, and then the application must be restarted and run within the container again, depending on the new image file. Since the data volume of at least one associated file is usually large, this results in slow image file loading and a lengthy application startup process.

[0004] As can be seen from the above, starting an application in a container by relying on an image file is less efficient. Summary of the Invention

[0005] This application provides a file processing method, system, cluster, apparatus, device, and program product to address the problem of low efficiency in starting applications in containers that rely on image files.

[0006] In a first aspect, embodiments of this application provide a document processing method, the method comprising:

[0007] Determine the target storage area where the image file is located, wherein the image file is a file obtained by packaging at least one associated file that the target application depends on at runtime;

[0008] Based on the target storage area, the image file is mounted to the container used to run the target application.

[0009] In one possible implementation, the container runs on a target node in the cluster, and the target storage area is a block device mounted on the target node;

[0010] The image file is retrieved from the image repository and then pulled to the block device via an image pre-puller after being uploaded to the image repository by the user.

[0011] In one possible implementation, the image file is specifically pulled to the block device in the following manner:

[0012] After establishing the connection between the image repository and the image pre-puller, the image file is pulled from the image repository through the image pre-puller, and it is determined whether there is a mounted block device in the target node;

[0013] If it exists, the pulled image file will be stored in the mounted block device;

[0014] If it does not exist, the block device management component is invoked to create a new block device, and the newly created block device is mounted on the target node. The image file is then saved in the newly created block device.

[0015] In one possible implementation, the image file is obtained in the following manner:

[0016] Add the associated files that the target application depends on at runtime, as well as the corresponding auxiliary files, to the preset directory;

[0017] A manifest file is generated in the preset directory. The manifest file includes the hash values ​​corresponding to the associated files and auxiliary files, as well as the metadata corresponding to the associated files.

[0018] The system packages the files in the preset directory, calculates the hash value of the packaged files, and generates the corresponding image file based on the packaged files and the calculated hash value.

[0019] In one possible implementation, the file processing method is applied to the Container Storage Interface (CSI) driver plugin in the target node; determining the target storage area where the image file is located includes:

[0020] Get the volume mount request sent by the node management component;

[0021] Based on the volume mount request, determine the target storage area where the image file is located.

[0022] In one possible implementation, the target node includes multiple CSI driver plugins;

[0023] The target CSI driver plugin is determined by the node management component from among the plurality of CSI driver plugins based on the persistent volume declaration PVC information;

[0024] The persistent volume declaration information is generated by the CSI provisioner based on the persistent volume declaration request after it receives the user's input request for configuring associated files for the target application.

[0025] In one possible implementation, mounting the image file to a container for running the target application, based on the target storage region, includes:

[0026] Based on the target storage area, the image file is mounted as a data volume to the container used to run the target application via volume mounting.

[0027] In one possible implementation, the volume mount request includes the target address of the image file in the image repository and the target directory in the container where the image file is to be mounted;

[0028] Based on the target storage area, the image file is mounted as a data volume to the container used to run the target application via volume mounting, including:

[0029] Determine whether the target address exists in the target storage area;

[0030] If it exists, then the target storage area is mounted to the target directory.

[0031] In one possible implementation, the method further includes:

[0032] If the target address does not exist in the target storage area, the target address is created in the target storage area, and the image file is pulled from the image repository to the target storage area according to the target address;

[0033] After the fetch is complete, the target storage area is mounted to the target directory.

[0034] In one possible implementation, if the volume mount request includes an update instruction, the method further includes, before mounting the target storage region to the target directory:

[0035] Determine whether the image file corresponding to the target address in the target storage area is consistent with the image file corresponding to the target address in the image repository;

[0036] If they match, a message indicating that an image file already exists will be returned.

[0037] If they are inconsistent, the image file corresponding to the target address is pulled from the image repository, and the image file in the target storage area is updated to the pulled image file.

[0038] In one possible implementation, the volume mount request includes a replacement strategy to mount the target storage region to the target directory, including:

[0039] If other data volumes are mounted under the target directory, then according to the replacement strategy, it is determined whether to replace the other data volumes with the target storage area;

[0040] If so, the target storage area is mounted to the target directory.

[0041] In one possible implementation, the file processing method is applied to a CSI driver plugin in a target node, the CSI driver plugin including multiple mounters, each of which corresponds one-to-one with a multiple runtime component in the target node:

[0042] Mounting the image file as a data volume to the container used to run the target application involves:

[0043] Determine the type of the container runtime component;

[0044] Depending on the type of the container runtime component, the corresponding mounter is invoked to perform a volume mount operation, thereby mounting the image file to the container.

[0045] In one possible implementation, the target application is used to run in multiple containers, with the same target storage region mounted to the multiple containers; or...

[0046] The target application shares the image file with other applications in the cluster, and the target storage area storing the image file is mounted to the container used to run the target application and the container used to run other applications.

[0047] Secondly, embodiments of this application provide a file processing method applied to a mirror pre-fetcher, the method comprising:

[0048] Pull image files from the image repository, wherein the image file is a file obtained by packaging at least one associated file that the target application depends on at runtime;

[0049] The image file is stored in the target storage area and is used to mount the container for running the target application.

[0050] Thirdly, embodiments of this application provide a file processing method applied to a node management component, the method comprising:

[0051] Retrieve persistent volume declaration information;

[0052] Based on the persistent volume declaration information, a volume mount request is determined and sent to the corresponding CSI driver plugin. The volume mount request is used to instruct the CSI driver plugin to determine the target storage area corresponding to the image file, and to mount the image file to the container used to run the target application based on the target storage area.

[0053] The image file is a file obtained by packaging at least one associated file that the target application depends on at runtime.

[0054] Fourthly, embodiments of this application provide a file processing method applied to a CSI pre-configurator, the method comprising:

[0055] Obtain the persistent volume declaration request from the user, which is used to configure associated files for the target application;

[0056] Based on the persistent volume declaration request, a corresponding volume creation request is determined and sent to the CSI driver plugin. The volume creation request is used to instruct the CSI driver plugin to determine the target storage area corresponding to the image file and call the image pre-fetcher to store the image file in the target storage area. The image file is a file obtained by packaging at least one associated file that the target application depends on at runtime. The image file is used to mount to the container used to run the target application.

[0057] Fifthly, embodiments of this application provide a file processing system, including: a CSI driver plugin, an image pre-puller, a node management component, and a CSI provisioner;

[0058] Wherein, the CSI driver plugin is used to execute the method described in any one of the first aspects;

[0059] The mirror pre-puller is used to perform the method described in any of the second aspects;

[0060] The node management component is used to perform the method described in any of the third aspects;

[0061] The CSI prefitter is used to perform the method described in any of the fourth aspects.

[0062] In a sixth aspect, embodiments of this application provide a cluster including a first node, at least one second node and a third node. The first node is equipped with an image pre-puller, the second node is equipped with a node management component and a CSI driver plugin, and has a corresponding target storage area for storing image files. The third node is equipped with a CSI provisioner.

[0063] Wherein, the CSI driver plugin is used to execute the method described in any one of the first aspects;

[0064] The mirror pre-puller is used to perform the method described in any of the second aspects;

[0065] The node management component is used to perform the method described in any of the third aspects;

[0066] The CSI prefitter is used to perform the method described in any of the fourth aspects.

[0067] Seventhly, embodiments of this application provide a file processing apparatus, the apparatus comprising: a determining module and a mounting module, wherein,

[0068] The determining module is used to determine the target storage area where the image file is located, wherein the image file is a file obtained by packaging at least one associated file that the target application depends on at runtime;

[0069] The mounting module is used to mount the image file to a container for running the target application, based on the target storage area.

[0070] In one possible implementation, the container runs on a target node in the cluster, and the target storage area is a block device mounted on the target node;

[0071] The image file is retrieved from the image repository and then pulled to the block device via an image pre-puller after being uploaded to the image repository by the user.

[0072] In one possible implementation, the image file is specifically pulled to the block device in the following manner:

[0073] After establishing the connection between the image repository and the image pre-puller, the image file is pulled from the image repository through the image pre-puller, and it is determined whether there is a mounted block device in the target node;

[0074] If it exists, the pulled image file will be stored in the mounted block device;

[0075] If it does not exist, the block device management component is invoked to create a new block device, and the newly created block device is mounted on the target node. The image file is then saved in the newly created block device.

[0076] In one possible implementation, the image file is obtained in the following manner:

[0077] Add the associated files that the target application depends on at runtime, as well as the corresponding auxiliary files, to the preset directory;

[0078] A manifest file is generated in the preset directory. The manifest file includes the hash values ​​corresponding to the associated files and auxiliary files, as well as the metadata corresponding to the associated files.

[0079] The system packages the files in the preset directory, calculates the hash value of the packaged files, and generates the corresponding image file based on the packaged files and the calculated hash value.

[0080] In one possible implementation, the file processing method is applied to the Container Storage Interface (CSI) driver plugin in the target node; the determining module is specifically used for:

[0081] Get the volume mount request sent by the node management component;

[0082] Based on the volume mount request, determine the target storage area where the image file is located.

[0083] In one possible implementation, the target node includes multiple CSI driver plugins;

[0084] The target CSI driver plugin is determined by the node management component from among the plurality of CSI driver plugins based on the persistent volume declaration PVC information;

[0085] The persistent volume declaration information is generated by the CSI provisioner based on the persistent volume declaration request after it receives the user's input request for configuring associated files for the target application.

[0086] In one possible implementation, the mounting module is specifically used for:

[0087] Based on the target storage area, the image file is mounted as a data volume to the container used to run the target application via volume mounting.

[0088] In one possible implementation, the volume mount request includes the target address of the image file in the image repository and the target directory of the image file to be mounted in the container; the mount module is specifically used for:

[0089] Determine whether the target address exists in the target storage area;

[0090] If it exists, then the target storage area is mounted to the target directory.

[0091] In one possible implementation, the mounting module is further used for:

[0092] If the target address does not exist in the target storage area, the target address is created in the target storage area, and the image file is pulled from the image repository to the target storage area according to the target address;

[0093] After the fetch is complete, the target storage area is mounted to the target directory.

[0094] In one possible implementation, if the volume mount request includes an update instruction, the mount module is further configured to:

[0095] Determine whether the image file corresponding to the target address in the target storage area is consistent with the image file corresponding to the target address in the image repository;

[0096] If they match, a message indicating that an image file already exists will be returned.

[0097] If they are inconsistent, the image file corresponding to the target address is pulled from the image repository, and the image file in the target storage area is updated to the pulled image file.

[0098] In one possible implementation, the volume mount request includes a replacement strategy, and the mount module is specifically used for:

[0099] If other data volumes are mounted under the target directory, then according to the replacement strategy, it is determined whether to replace the other data volumes with the target storage area;

[0100] If so, the target storage area is mounted to the target directory.

[0101] In one possible implementation, the file processing method is applied to a CSI driver plugin in a target node. The CSI driver plugin includes multiple mounters, each corresponding one-to-one with a multiple runtime component in the target node. The mounting module is specifically used for:

[0102] Determine the type of the container runtime component;

[0103] Depending on the type of the container runtime component, the corresponding mounter is invoked to perform a volume mount operation, thereby mounting the image file to the container.

[0104] In one possible implementation, the target application is used to run in multiple containers, with the same target storage region mounted to the multiple containers; or...

[0105] The target application shares the image file with other applications in the cluster, and the target storage area storing the image file is mounted to the container used to run the target application and the container used to run other applications.

[0106] Eighthly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0107] The memory stores computer-executed instructions;

[0108] The processor executes computer execution instructions stored in the memory, causing the processor to perform the method described in any one of the first to fourth aspects.

[0109] Ninthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in any one of the first to fourth aspects.

[0110] In a tenth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method shown in any one of the first to fourth aspects.

[0111] This application provides a file processing method, system, cluster, apparatus, device, and program product. The CSI driver plugin can determine the target storage area where the image file is located and, based on the target storage area, mount the image file to a container used to run the target application. Because the technical solution of this application can decouple at least one associated file from at least one application file, package at least one associated file with a large amount of data into an image file, and mount the image file to the container running the target application, it eliminates the need to copy the image file, achieving preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting applications in containers that rely on image files. Attached Figure Description

[0112] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0113] Figure 1 A schematic diagram illustrating an application scenario provided for an exemplary embodiment of this application;

[0114] Figure 2 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 1 ;

[0115] Figure 3 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 2 ;

[0116] Figure 4 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 3 ;

[0117] Figure 5 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 4 ;

[0118] Figure 6 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 5 ;

[0119] Figure 7 A schematic diagram illustrating a file processing method provided for an exemplary embodiment of this application;

[0120] Figure 8 The comparison chart provided for this application shows the effect of using this solution with other solutions;

[0121] Figure 9 A schematic diagram of a document processing system provided for an exemplary embodiment of this application;

[0122] Figure 10 A schematic diagram of the cluster structure provided for an exemplary embodiment of this application;

[0123] Figure 11 This is a schematic diagram of the structure of a document processing device provided in an embodiment of this application;

[0124] Figure 12 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. Detailed Implementation

[0125] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0126] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0127] Below, in conjunction with Figure 1 The application scenarios of this application will be explained.

[0128] Figure 1 This is a schematic diagram illustrating an application scenario provided for an exemplary embodiment of this application. Please refer to [link / reference]. Figure 1 A cluster can run multiple containers. A cluster can be a cluster of running containers. For example, a cluster can be a Kubernetes (Container Orchestration Engine) cluster, where multiple containers can be named container C1, container C2, ...

[0129] For any given container, an application can run within it. For example, application APP1 can run in container C1.

[0130] Image files can be stored in the target storage area. Image files can be mounted as data volumes into containers using volume mounting. Applications can run within containers that depend on image files. For example, in container C1, application APP1 can run by depending on the image file.

[0131] In related technologies, at least one associated file and at least one application file are typically packaged to generate an image file. This image file can then be loaded, and the application can be started and run within a container based on it. However, in this approach, if at least one associated file or at least one application file needs to be updated, a new image file must be generated, loaded, and then the application must be restarted and run within the container again, depending on the new image file. Since at least one associated file is usually large, this results in slow image file loading and a lengthy application startup process. Therefore, starting an application within a container based on an image file is inefficient.

[0132] In this embodiment, at least one associated file can be decoupled from at least one application file. The at least one associated file is packaged into an image file, which is then stored in a target storage area. The image file can then be mounted into the container running the target application based on the target storage area, eliminating the need to copy the image file. This achieves preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting applications in containers that rely on image files.

[0133] The technical solutions shown in this application will now be described in detail through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other; for identical or similar content, the description will not be repeated in different embodiments.

[0134] Figure 2 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 1 Please see. Figure 2 The method may include:

[0135] S201. Determine the target storage area where the image file is located.

[0136] An image file can be a file that is packaged to contain at least one associated file that the target application depends on at runtime.

[0137] Association files can be large-scale data files. For example, an association file can be a model file.

[0138] Optionally, the image file can be obtained by: adding the associated files that the target application depends on at runtime and the corresponding auxiliary files to the preset directory; generating a manifest file in the preset directory; packaging the files in the preset directory, calculating the hash value of the packaged files, and generating the corresponding image file based on the packaged files and the calculated hash value.

[0139] In response to the user's initial creation operation, the user device can create a preset directory. After the preset directory is created, the user device can, in response to the user's add operation, add the associated files that the target application depends on at runtime, as well as the corresponding auxiliary files, to the preset directory. For example, the auxiliary files may include dependency library files and / or configuration files.

[0140] In response to a user's second creation action, the user device can create a manifest file in a preset directory. The manifest file may include the hash values ​​corresponding to associated files and auxiliary files, as well as the metadata corresponding to the associated files.

[0141] A hash value is the result of a hash calculation performed on a file. Hash values ​​can be used to verify the integrity of a file, determining whether it has been tampered with.

[0142] For example, the manifest file may include at least one of the following: the version of the manifest file, the data format, information about at least one mirror layer, and metadata corresponding to the associated file.

[0143] Associated files and auxiliary files can be distributed in the various image layers of the manifest file according to the logic of building the image. Therefore, the information of any image layer can include the data format, hash value, and file size of the file corresponding to that image layer.

[0144] For example, in the manifest file, image layer 1 can be the image layer corresponding to the associated file. The information of image layer 1 can include the data format of the associated file as Tape Archive (TAR) format, the hash value of the associated file as 123456abcdef, and the file size of the associated file as 20GB.

[0145] For example, if the associated file is a model file, the metadata corresponding to the associated file may include the model name, creation time, etc.

[0146] In response to the user-inputted TAR command, the user device can package files (i.e., associated files, corresponding auxiliary files, and manifest files) in a preset directory according to the Open Container Initiative (OCI) standard, calculate the hash value of the packaged files, and generate a corresponding image file based on the packaged files and the calculated hash value. The image file can include associated files, auxiliary files, and the manifest file. The data format of the image file can be an OCI-compatible format.

[0147] After obtaining the image file, the user device can use an OCI-compatible container management tool to upload the image file to the image repository.

[0148] Image files can be pulled from the image repository to the block device via the image pre-puller after the user uploads them to the image repository. The target storage area can be a block device mounted on the target node, and the block device can be a storage device such as a disk, hard disk drive (HDD), or solid state disk (SSD). The target node can be a physical node in the cluster (e.g., the host machine) or a virtual node in the cluster (e.g., a virtual machine).

[0149] For example, if the target node is the host machine host1, the target storage area can be a disk mounted on the host machine; if the target node is a virtual machine VM1, the target storage area can be a virtual disk mounted on the virtual machine.

[0150] Optionally, the image file can be pulled to the block device in the following way: After establishing a connection between the image repository and the image pre-puller, the image file is pulled from the image repository through the image pre-puller, and it is determined whether there is a mounted block device on the target node; if there is, the pulled image file is stored in the mounted block device; if there is, the block device management component is called to create a new block device, and the newly created block device is mounted on the target node, and the image file is saved in the newly created block device.

[0151] Optionally, the cluster can include an image repository and an image pre-puller. The image repository can be the storage space on a storage server used to store image files. The image pre-puller is used to pull image files.

[0152] The image pre-puller can obtain the network address and key of the image repository, and establish a remote call connection with the image repository within a preset connection time based on the network address and key.

[0153] It should be noted that if the connection creation fails, the image pre-pull can return a failure message to the user, which can be used to indicate that the connection with the image repository failed.

[0154] After establishing a connection between the image repository and the image pre-puller, image files can be pulled from the image repository using the image pre-puller. Once the image files are pulled, the image pre-puller saves them to the block device mounted on the target node.

[0155] Optionally, the image pre-pull saves the image file to the block device mounted on the target node, which can include the following two cases:

[0156] Scenario 1: There are already mounted block devices in the target node.

[0157] In this case, the image pre-puller can store the pulled image file into the mounted block device.

[0158] For example, if the target node is host1, the image pre-fetcher determines that there is a mounted block device 1 on host1, and then the image pre-fetcher can store the pulled image file to block device 1.

[0159] Scenario 2: There are no mounted block devices in the target node.

[0160] In this scenario, the image prefetcher can invoke the block device management component to create a new block device and mount it onto the target node. After successful mounting, the image prefetcher can save the image file within the newly created block device.

[0161] Optionally, a block device management component can be included in the cluster. The block device management component can be used to create block devices, mount block devices, etc.

[0162] For example, if the target node is host1, and the image pre-fetcher determines that there is no mounted block device on host1, the image pre-fetcher can call the block device management component to create block device 2 and mount block device 2 to host1. After successful mounting, the image pre-fetcher can store the pulled image file to block device 2.

[0163] The target node can have a target Container Storage Interface (CSI) driver plugin. The target CSI driver plugin can respond to volume mount requests and determine the target storage region where the image file is located.

[0164] For example, the target CSI driver plugin can respond to volume mount request 1 and determine that the target storage area where the image file is located is block device 2.

[0165] S202. Mount the image file to the container used to run the target application, based on the target storage area.

[0166] Optionally, the CSI driver plugin can mount the image file as a data volume to the container used to run the target application by mounting the volume, depending on the target storage area.

[0167] Data volumes can be used to store data. In containerized environments, data volumes enable persistent storage, allowing data to remain unchanged when containers stop or restart; they also enable data sharing, allowing multiple containers to access the same data volume simultaneously; and they provide flexible storage, enabling containers to dynamically mount different data volumes, allowing containers to access data in different data volumes or store data in different data volumes.

[0168] For example, if the target storage area is block device 2, and if application APP1 is running in container C1, the mounter can mount the image file as a data volume to container C1 through volume mounting, so that application APP1 can run based on the image file.

[0169] In this embodiment, the CSI driver plugin can determine the target storage area where the image file is located, and mount the image file to the container used to run the target application based on the target storage area. Because the technical solution of this application can decouple at least one associated file from at least one application file, package at least one associated file with a large amount of data into an image file, and mount the image file as a data volume to the container running the target application via volume mounting, there is no need to copy the image file. This achieves preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting applications in containers that rely on image files.

[0170] The file processing method described above can be applied to the CSI driver plugin in the target node. Below, in Figure 2 Based on the illustrated embodiments, combined with Figure 3 The above document processing methods will be explained in detail.

[0171] Figure 3 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 2 Please see. Figure 3 The methods may include:

[0172] S301. Obtain the volume mount request sent by the node management component.

[0173] Optionally, the target node may include a node management component and multiple CSI driver plugins. The target CSI driver plugin may be determined by the node management component from multiple CSI driver plugins based on the persistent volume declaration PVC information.

[0174] Specifically, the node management component can determine the Persistent Volume Claim (PVC) identifier in the container configuration information of the container running the target application, and determine the PVC information based on the PVC identifier. Then, it can determine the target CSI driver plugin corresponding to the PVC information from multiple CSI driver plugins based on the PVC information, and send a volume mount request to the target CSI driver plugin.

[0175] PVC information is generated by the CSI provisioner based on the user's PVC request, which is used to configure associated files for the target application. PVC information may include storage capacity, access mode, and storage class identifier. The storage class identifier indicates the corresponding storage class.

[0176] A cluster can include multiple storage categories. For any given storage category, the following can be defined: storage type, storage configuration parameters, and CSI driver plugin identifier.

[0177] Optionally, the node management component can determine the target storage category based on the storage category identifier in the PVC information, and determine the target CSI driver plugin corresponding to the PVC information among multiple CSI driver plugins based on the CSI driver plugin identifier in the target storage category.

[0178] The node management component can determine the target storage region information bound to the PVC information. The target storage region information may include at least one of the following: the identifier of the target storage region, the access mode of the target storage region, the storage capacity of the target storage region, and the target address of the image file in the image repository.

[0179] The node management component can also determine the target directory where the image file should be mounted in the container running the target application by looking at the container configuration information of the container running the target application.

[0180] The node management component can generate a volume mount request based on the information of the target storage area and the target directory of the image file to be mounted in the container running the target application, and send the volume mount request to the target CSI driver plugin.

[0181] The volume mount request may include the target directory in the container where the image file is to be mounted, the target address of the image file in the image repository, and the identifier of the target storage area.

[0182] For example, a volume mount request may include the target address as address A, the target directory as " / data directory", and the identifier of the target storage region as BD2.

[0183] S302. Determine the target storage area where the image file is located based on the volume mount request.

[0184] Because the volume mount request includes the identifier of the target storage region, the target CSI driver plugin can determine the target storage region where the image file is located among multiple block devices mounted on the target node, based on the identifier of the target storage region.

[0185] For example, if the target storage region is identified as BD2, and the target node has block device 1, block device 2, and block device 3 mounted, the target CSI driver plugin can determine that block device 2 is the target storage region based on the target storage region identifier BD2.

[0186] S303. Determine whether the target directory exists in the directory corresponding to the container running the target application.

[0187] Since the volume mount request can include the target directory of the image file to be mounted in the container, the target CSI driver plugin can determine whether the target directory exists in the directory corresponding to the container running the target application. If it does not exist, step S304 can be executed; if it exists, step S305 can be executed.

[0188] For example, if the container running the target application is container C1 and the target directory is " / data directory", the target CSI driver plugin can determine whether the " / data directory" exists in the directory corresponding to container C1. If it does not exist, step S304 can be executed; if it exists, step S305 can be executed.

[0189] S304. Create the target directory in the directory corresponding to the container running the target application.

[0190] For example, if the container running the target application is container C1 and the target directory is " / data directory", if the target CSI driver plugin determines that the " / data directory" does not exist in the directory corresponding to container C1, the target CSI driver plugin can create the " / data directory" in the directory corresponding to container C1.

[0191] After completing step S304, step S305 can be executed.

[0192] S305. Determine whether the target address exists in the target storage area.

[0193] If the target directory exists in the directory corresponding to the container running the target application, and since the volume mount request includes the target address of the image file in the image repository, the target CSI driver plugin can further determine whether the target address exists in the target storage area. If it exists, it means that the image file exists in the target storage area, and step S306 can be executed; if it does not exist, it means that the image file does not exist in the target storage area, and step S309 can be executed.

[0194] For example, if the target address is address A and the target storage area is block device 2, the target CSI driver plugin can determine whether address A exists in block device 2. If it exists, it means that an image file exists in block device 2, and step S306 can be executed; if it does not exist, it means that an image file does not exist in block device 2, and step S309 can be executed.

[0195] S306. Determine whether the image file corresponding to the target address in the target storage area is consistent with the image file corresponding to the target address in the image repository.

[0196] Optionally, if the volume mount request includes an update instruction, the target CSI driver plugin can determine whether the image file corresponding to the target address in the target storage region is consistent with the image file corresponding to the target address in the image repository. If they are consistent, step S307 can be executed; if they are inconsistent, step S308 can be executed.

[0197] For example, if the target address is address A, the target storage area is block device 2, and the image file corresponding to address A in block device 2 is image file 1.0, and the image file corresponding to address A in the image repository is image file 1.0, then the target CSI driver plugin can determine that the image file corresponding to address A in block device 2 is consistent with the image file corresponding to address A in the image repository, and then step S307 can be executed.

[0198] If the image file corresponding to address A in block device 2 is image file 1.0, and the image file corresponding to address A in the image repository is image file 2.0, then the target CSI driver plugin can determine that the image file corresponding to address A in block device 2 is inconsistent with the image file corresponding to address A in the image repository, and then step S308 can be executed.

[0199] S307. Returns a message indicating that an image file already exists.

[0200] If the image file corresponding to the target address in the target storage area is the same as the image file corresponding to the target address in the image repository, the target CSI driver plugin can return a prompt message indicating that the image file already exists. After returning the prompt message indicating that the image file already exists, step S310 can be executed.

[0201] For example, if the target address is address A and the target storage area is block device 2, if the target CSI driver plugin determines that the image file corresponding to address A in block device 2 is consistent with the image file corresponding to address A in the image repository, then it returns a prompt message indicating that the image file already exists.

[0202] S308. Pull the image file corresponding to the target address from the image repository, and update the image file in the target storage area with the pulled image file.

[0203] If the image file corresponding to the target address in the target storage area is inconsistent with the image file corresponding to the target address in the image repository, an update operation needs to be performed on the image file corresponding to the target address in the target storage area. That is, the target CSI driver plugin can pull the image file corresponding to the target address from the image repository and update the image file in the target storage area with the pulled image file. After updating to the pulled image file, step S310 can be executed.

[0204] For example, if the target address is address A, the target storage area is block device 2, and the image file corresponding to address A in block device 2 is image file 1.0, and the image file corresponding to address A in the image repository is image file 2.0, then the target CSI driver plugin can pull the image file 2.0 corresponding to address A from the image repository and update the image file 1.0 in block device 2 to the pulled image file 2.0.

[0205] S309. Create a target address in the target storage area, and pull the image file from the image repository to the target storage area according to the target address.

[0206] If the target address does not exist in the target storage area, it means that the image file does not exist in the target storage area. In this case, the target CSI driver plugin can create the target address in the target storage area and pull the image file from the image repository to the target storage area according to the target address. After the pull is complete, step S310 can be executed.

[0207] For example, if the target address is address A and the target storage area is block device 2, if address A does not exist in block device 2, it means that there is no image file in block device 2. In this case, the target CSI driver plugin can create address A in block device 2 and pull the image file corresponding to address A from the image repository to block device 2 based on address A.

[0208] S310. Mount the target storage area in the target directory.

[0209] Mounting a target storage region to the target directory can include the following two scenarios:

[0210] Scenario 1: No other data volumes are mounted in the target directory.

[0211] In this case, the target CSI driver plugin can mount the target storage area in the target directory to mount the image file.

[0212] For example, if the target directory is " / data directory" and no other data volumes are mounted under " / data directory", and the target storage area is block device 2, then the target CSI driver plugin can mount block device 2 under " / data directory", and block device 2 includes image files.

[0213] Scenario 2: Other data volumes are mounted in the target directory.

[0214] In this scenario, if the volume mount request includes a replacement policy, the target CSI driver plugin can determine whether to replace the target storage region with another data volume based on the replacement policy. If so, the target storage region is mounted to the target directory.

[0215] For example, if the target directory is " / data directory", and data volume V1 is mounted under " / data directory", and the target storage area is block device 2, if the replacement policy indicates that other data volumes under the target directory should be replaced with the target storage area, then the target CSI driver plugin can replace data volume V1 with block device 2 under " / data directory", and block device 2 includes image files.

[0216] It should be noted that the target application can run in multiple containers. Steps S301-S310 allow the same target storage area to be mounted to multiple containers. Alternatively, the target application can share an image file with other applications in the cluster. Steps S301-S310 allow the target storage area storing the image file to be mounted to both the container running the target application and the container running other applications. Multiple containers can directly utilize the pre-loaded image file and can mount the same image file via volume mounting, significantly reducing data loading time in large-scale container scenarios.

[0217] It should be noted that, in Figure 3 The various processing steps (S301-S310) shown in the embodiments do not constitute a specific limitation on the file processing procedure. In other embodiments of this application, the file processing procedure may include more than Figure 3 The embodiments may include more or fewer steps. For example, a document processing procedure may include... Figure 3 Some steps in the embodiments, or, Figure 3 Some steps in the embodiments can be replaced by steps with the same function, or Figure 3 Some steps in the embodiments can be broken down into multiple steps, etc.

[0218] In this embodiment, the target CSI driver plugin can obtain the volume mount request sent by the node management component and determine the target storage area where the image file is located based on the volume mount request. The target CSI driver plugin can determine whether a target directory exists in the directory corresponding to the container running the target application. If it does not exist, the target directory can be created in the directory corresponding to the container running the target application; if it exists, the target address can be determined to exist in the target storage area. If a target address exists in the target storage area, it is further determined whether the image file corresponding to the target address in the target storage area is consistent with the image file corresponding to the target address in the image repository; if they are consistent, a prompt message indicating that an image file already exists can be returned, and the target storage area can be mounted under the target directory; if they are inconsistent, the image file corresponding to the target address can be pulled from the image repository, and the image file in the target storage area can be updated to the pulled image file, and then the target storage area can be mounted under the target directory. If a target address does not exist in the target storage area, a target address can be created in the target storage area, and the image file can be pulled from the image repository to the target storage area based on the target address, and then the target storage area can be mounted under the target directory. In this application, at least one associated file can be decoupled from at least one application file. At least one associated file with a large amount of data can be packaged into an image file and mounted as a data volume into the container running the target application by volume mounting. This eliminates the need to copy the image file and achieves preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting the application in the container that depends on the image file.

[0219] The above Figure 2 and Figure 3 The examples described are all file processing methods executed on the CSI driver plugin side. Correspondingly, the CSI driver plugin can also unload image files. Specifically, the CSI driver plugin can receive a volume unloading request sent by the user. The volume unloading request may include the identifier of the target storage area and the target directory. The CSI driver plugin can determine whether the target directory is a mount point and verify its mount status. If the mount status of the target directory is mounted, the CSI driver plugin can perform an unloading operation to unload the target storage area from the target container, ensuring that the target storage area is correctly released.

[0220] It should be noted that if errors are encountered during the uninstallation process, the CSI driver plugin can provide clear error feedback and count and record the errors.

[0221] Below, in conjunction with Figure 4 This section explains the file processing methods executed on the image prefetcher side; combined with... Figure 5 This section explains the file processing methods executed by the node management component; combined with... Figure 6The document processing method executed on the CSI pre-fitter side is explained.

[0222] Figure 4 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 3 Please see. Figure 4 The method may include:

[0223] S401. Pull image files from the image repository.

[0224] Optionally, the cluster can include an image repository and an image pre-puller. The image repository can be storage space on a storage server used to store image files. The image pre-puller is used to pull image files. The image file is a packaged file obtained by packaging at least one associated file that the target application depends on at runtime.

[0225] The image pre-puller can obtain the network address and key of the image repository, and establish a remote call connection with the image repository within a preset connection time based on the network address and key.

[0226] After establishing a connection between the image repository and the image pre-puller, the image pre-puller can pull image files from the image repository.

[0227] Optionally, the image pre-puller can pull image files from the image repository in two ways: synchronous image pulling and asynchronous image pulling.

[0228] In the synchronous image pull method, the image pre-puller can specifically be a synchronous image pre-puller. The synchronous image pre-puller is responsible for synchronization operations related to the remote container image, such as pulling the image and obtaining its size.

[0229] In asynchronous image fetching, the image prefetcher can specifically be an asynchronous image prefetcher. The asynchronous image prefetcher provides the ability to operate asynchronously, which can be used to handle image fetch tasks that may take a long time to complete. The asynchronous image prefetcher can maintain and manage fetch sessions, ensuring efficient handling of multiple image fetch requests even under high concurrency. The asynchronous image prefetcher allows starting fetch sessions and waiting for them to complete or time out. Optionally, asynchronous fetching can be performed during off-peak hours, helping to balance network load.

[0230] For example, after establishing a connection between the image repository and the image pre-puller, the image pre-puller can pull image file 1.0 from the image repository.

[0231] S402. Store the image file in the target storage area.

[0232] The target storage region can be a block device mounted on a target node. The target node can be a physical node in the cluster (e.g., a host machine) or a virtual node in the cluster (e.g., a virtual machine).

[0233] Image files can be used to mount containers used to run the target application.

[0234] For example, if the target node is host1, and if host1 has a mounted block device 1, the image pre-puller can store the pulled image file 1.0 to block device 1. This allows the CSI driver plugin to mount the image file 1.0 as a data volume to container C1, which is used to run application APP1, based on block device 1, so that application APP1 can run based on the mounted image file 1.0.

[0235] Optionally, this application may support an image pre-fetching mechanism, which allows frequently used images to be pre-fetched and cached. The image pre-fetcher can improve overall download efficiency by pre-fetching and caching frequently used image files.

[0236] Image pre-warming can be implemented before mounting image files. After creating a persistent volume declaration / volume, the file processing system can call the image pre-puller to pre-pull the image files onto the corresponding nodes. When a container replica is instantiated, the container can directly utilize these pre-pulled image files, thus eliminating the pull delay when the image is first mounted to the container. Simultaneously, the file processing system can automatically pre-pull potentially needed image files based on historical data analysis (e.g., the usage frequency of each image file within a historical period), usage pattern prediction (e.g., user habits regarding image files), and updates to image files in the image repository. This pre-loading of image files reduces application startup time and improves the efficiency of starting applications that depend on image files within containers.

[0237] In this embodiment, the image pre-fetcher can pull image files from the image repository and store them in the target storage area. Because at least one associated file can be decoupled from at least one application file in this application, at least one associated file with a large data volume can be packaged into an image file. The image pre-fetcher then stores the image file in the target storage area, allowing the CSI driver plugin to mount the image file as a data volume into the container running the target application via volume mounting, eliminating the need to copy the image file. This preloading of at least one associated file accelerates the application startup process and improves the efficiency of starting applications in containers that rely on image files.

[0238] Figure 5 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 4 Please see. Figure 5 The method may include:

[0239] S501, Obtain PVC information.

[0240] PVC information is generated by the CSI configurer after it receives a PVC request from the user, which is used to configure associated files for the target application. The PVC request can be user-inputted and used to configure associated files for the target application.

[0241] PVC information may include storage capacity, access mode, and storage class identifier. The storage class identifier indicates the corresponding storage class. Access mode may include read-only mode or read-write mode.

[0242] A cluster can include multiple storage categories. For example, these multiple storage categories may include: storage categories based on network file systems, storage categories based on distributed storage systems, storage categories based on local disks, storage categories based on cloud storage services, etc.

[0243] For any given storage category, the following can be defined: storage type, storage configuration parameters, and CSI driver plugin identifier.

[0244] Optionally, the node management component can determine the target storage category based on the storage category identifier in the PVC information, and determine the CSI driver plugin corresponding to the PVC information from among multiple CSI driver plugins based on the CSI driver plugin identifier in the target storage category.

[0245] For example, the node management component can obtain PVC information 1, including: storage capacity 30GB, read-only mode, and storage category identifier SC-1.

[0246] S502. Based on the PVC information, determine the roll mounting request and send it to the corresponding CSI driver plugin.

[0247] Optionally, the node management component can determine the target storage category based on the storage category identifier in the PVC information, and determine the CSI driver plugin corresponding to the PVC information among multiple CSI driver plugins in the target node based on the CSI driver plugin identifier in the target storage category.

[0248] For example, if the storage category identifier in PVC information 1 is SC-1, the node management component can determine the target storage category as storage category 1 based on the storage category identifier SC-1. If storage category 1 includes a CSI driver plugin identifier of CSI-2, then based on the CSI driver plugin identifier CSI-2, the CSI driver plugin corresponding to PVC information 1 can be determined as CSI driver plugin 2 among multiple CSI driver plugins.

[0249] The node management component can determine the target directory where the image file is to be mounted in the container running the target application from the container configuration information of the container running the target application, generate a volume mount request, and then send the volume mount request to the corresponding CSI driver plugin.

[0250] The image file is a packaged file containing at least one associated file that the target application depends on at runtime. The volume mount request may include the target directory in the container where the image file will be mounted, the target address of the image file in the image repository, and the identifier of the target storage region. The volume mount request can be used to instruct the CSI driver plugin to determine the target storage region corresponding to the image file and, based on the target storage region, mount the image file to the container used to run the target application.

[0251] For example, if the container running the target application is container C1, the node management component can determine from the container configuration information of container C1 that the target directory for mounting image file 1.0 in container C1 is " / data directory", and generate a volume mount request. The volume mount request can include the target address as address A, the target directory as " / data directory", and the identifier BD2 of the target storage area. The node management component can send this volume mount request to CSI driver plugin 2. CSI driver plugin 2 can determine the target storage area corresponding to image file 1.0 based on the volume mount request, and mount image file 1.0 as a data volume to container C1 used to run application APP1, so that application APP1 can run based on the mounted image file 1.0.

[0252] In this embodiment, the node management component can obtain PVC information, determine a volume mount request based on the PVC information, and send it to the corresponding CSI driver plugin. Because at least one associated file can be decoupled from at least one application file in this application, at least one associated file with a large data volume can be packaged into an image file. The node management component then sends a volume mount request to the CSI driver plugin, enabling the CSI driver plugin to determine the target storage area corresponding to the image file based on the volume mount request. The image file is then mounted as a data volume into the container running the target application via volume mounting, eliminating the need to copy the image file. This achieves preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting applications in containers that rely on image files.

[0253] Figure 6 A flowchart illustrating a document processing method provided for an exemplary embodiment of this application. Figure 5 Please see. Figure 6 The method may include:

[0254] S601. Obtain the PVC request input by the user for configuring the associated file for the target application.

[0255] A PVC request may include storage capacity, access mode, storage category identifier, and the target address of the image file in the image repository. The storage category identifier indicates the corresponding storage category.

[0256] For example, the CSI provisioner can obtain PVC requests, which may include storage capacity of 30GB, access mode of read-only mode, storage category identifier of SC-1, etc.

[0257] S602. Based on the PVC request, determine the corresponding volume creation request and send it to the corresponding CSI driver plugin.

[0258] The target node can include multiple CSI driver plugins. The CSI provisioner can determine the target CSI driver plugin corresponding to the PVC request from among the multiple CSI driver plugins, and send a volume creation request to the target CSI driver plugin.

[0259] A cluster can include multiple storage categories. For any given storage category, the following can be defined: storage type, storage configuration parameters, and CSI driver plugin identifier.

[0260] Specifically, the CSI provisioner can determine the target storage class based on the storage class identifier in the PVC request, and then determine the CSI driver plugin corresponding to the PVC request from among multiple CSI driver plugins based on the CSI driver plugin identifier in the target storage class. For example, the CSI provisioner can determine that the CSI driver plugin corresponding to the PVC request is CSI driver plugin 2.

[0261] After the CSI provisioner determines the CSI driver plugin corresponding to the PVC request, the CSI provisioner can establish a remote call relationship with the CSI driver plugin corresponding to the PVC request and send a volume creation request to the CSI driver plugin.

[0262] A volume creation request can be generated based on a PVC request. The volume creation request can include storage capacity, access mode, and the target address of the image file in the image repository.

[0263] For example, the CSI provisioner can send a volume creation request to the CSI driver plugin 2. The volume creation request may include a storage capacity of 30GB, an access mode of read-only, and the target address of the image file in the image repository as address A.

[0264] A volume creation request can be used to instruct the CSI driver plugin to determine the target storage region corresponding to the image file and to invoke the image pre-puller to store the image file in the target storage region. The image file can then be used to mount the container used to run the target application.

[0265] After receiving a volume creation request, the CSI driver plugin can determine the storage region based on the request, create the target address within that storage region, and designate it as the target storage region corresponding to the image file. Once the target storage region for the image file is determined, the image pre-fetcher can be invoked to store the image file in the target storage region. The image file can be a file obtained by packaging at least one associated file that the target application depends on at runtime.

[0266] For example, after receiving the volume creation request, CSI driver plugin 2 can determine block device 2 based on the storage capacity of 30GB and the access mode of read-only in the volume creation request. It can then create address A within block device 2, thus identifying block device 2 as the target storage area corresponding to the image file. CSI driver plugin 2 can then invoke the image prefetcher to store the image file in block device 2.

[0267] The CSI driver plugin determines the target storage area corresponding to the image file based on the volume creation request, and after calling the image pre-fetcher to store the image file in the target storage area, it can bind the target storage area as a data volume with the PVC information generated based on the PVC request.

[0268] The CSI driver plugin can also be used to mount an image file as a data volume to a container used to run the target application, based on the target storage area, so that the target application can run based on the mounted image file.

[0269] It's important to note that the CSI provisioner not only triggers storage volume creation but also monitors the status of the CSI driver plugin, updating the PVC and PV statuses in the Kubernetes cluster to ensure users are aware of the current status of data volumes in real time. If an error occurs during storage operations, the CSI driver plugin can report the error information to the CSI provisioner, which can then update the PVC status to reflect problems in the provisioning process, thus achieving error handling and transparency.

[0270] In this embodiment, the CSI provisioner can obtain a PVC request input by the user for configuring associated files for the target application, and determine the corresponding volume creation request based on the PVC request and send it to the corresponding CSI driver plugin. Since the CSI provisioner can determine the corresponding volume creation request based on the PVC request and send it to the corresponding CSI driver plugin, the CSI driver plugin can determine the target storage area corresponding to the image file based on the volume creation request, and call the image pre-fetcher to store the image file in the target storage area. This allows the CSI driver plugin to subsequently determine the target storage area corresponding to the image file based on the volume mount request, and mount the image file as a data volume into the container running the target application via volume mount. This eliminates the need to copy the image file, achieving preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting applications in containers that rely on image files.

[0271] Below, based on any of the above embodiments, combined with Figure 7 The above document processing methods will be further explained.

[0272] Figure 7 This is a schematic diagram illustrating a file processing method provided for an exemplary embodiment of this application. Please refer to [link / reference]. Figure 7 It includes three processes: generating the image file, pulling and saving the image file, and mounting the image file.

[0273] (1) The process of generating the image file is explained below:

[0274] The process of generating the image file can be performed on the user's device and may include steps 1.1 to 1.5.

[0275] In step 1.1, the user equipment may create a preset directory in response to the user's first creation operation.

[0276] After creating the preset directory, in step 1.2, the user device can respond to the user's addition operation by adding the associated files, dependency library files, and configuration files that the target application depends on at runtime to the preset directory.

[0277] In step 1.3, the user equipment may, in response to the user's second creation operation, create a manifest file in a preset directory. The manifest file may include the hash values ​​corresponding to each file in the preset directory, as well as the metadata corresponding to the associated files.

[0278] In step 1.4, the user equipment can respond to the user-inputted TAR command, package a preset directory according to the OCI specification, calculate the hash value of the packaged files, and generate a corresponding image file based on the packaged files and the calculated hash value. The image file may include associated files, dependency library files, configuration files, and manifest files. The data format of the image file can be an OCI-compatible format.

[0279] In step 1.5, after obtaining the image file, the user device can use an OCI-compatible container management tool to upload the image file to the image repository. The image repository can store multiple image files.

[0280] (2) The process of pulling and saving the image file is described below:

[0281] The process of pulling and saving the image file can be performed in the cluster and may include steps 2.1 to 2.3.

[0282] In step 2.1, the image pre-puller can obtain the network address and key of the image repository, and establish a remote call connection with the image repository within a preset connection duration based on the network address and key.

[0283] After establishing a connection between the image repository and the image pre-puller, in step 2.2, the image pre-puller can pull image files from the image repository.

[0284] In step 2.3, after the image file is retrieved, the image pre-fetcher saves the image file to the block device mounted on the target node.

[0285] Specifically, if a block device already exists on the target node, the image pre-fetcher can store the image files it pulls into the mounted block device. If no block device exists on the target node, the image pre-fetcher can call the block device management component to create a new block device and mount the new block device onto the target node. After successful mounting, the image pre-fetcher can save the image files in the newly created block device.

[0286] For example, the image prefetcher saves the image file to the block device mounted on the target node.

[0287] (3) The process of mounting the image file is explained below:

[0288] The process of mounting the image file can be performed in the cluster and may include steps 3.1 to 3.4.

[0289] The target node may include node management components, CSI node components, and workload container groups (Pods).

[0290] The node management components can include a plugin manager, a runtime manager, and a volume manager. The plugin manager is responsible for discovering and loading various plugins in the cluster and managing the entire lifecycle of plugins, including starting, stopping, and unloading them. The runtime manager is used to create and start containers based on workload container group (Pod) definitions, and can interact with the container runtime, passing container configuration information from the workload container group (Pod) to the container runtime. The volume manager is used for managing data volumes, including provisioning, allocating, mounting, and unmounting.

[0291] The CSI node component may include a node driver registrar and at least one CSI driver plugin. For example, at least one CSI driver plugin may be CSI driver plugin 1 and CSI driver plugin 2.

[0292] The node driver registrar can be used to register CSI node components, create and maintain socket files pointing to CSI node components on each node, and periodically perform health checks on CSI node plugins to ensure their service availability and stability. The socket files can be used to establish communication connections between CSI node components and other components (such as node management components).

[0293] When registering CSI node components, the node driver registrar sets necessary environment variables and handles version negotiation, ensuring compatibility between the CSI node components and the Kubernetes cluster. The node driver registrar provides transparency regarding the operation of CSI node components through logging, allowing any registration errors or communication problems to be detected and resolved promptly. The node driver registrar is crucial for the lifecycle management of CSI node components; it not only participates in plugin deployment and operation but also provides recovery support in case of plugin failure.

[0294] CSI driver plugins bridge the communication gap between the cluster and the underlying storage system, enabling the cluster to easily manage and use various types of underlying storage resources, providing persistent storage services for containerized applications. For example, CSI driver plugins can be used to create, mount, and unmount volumes, and monitor and maintain the status of volumes.

[0295] CSI driver plugins can include various types, such as CSI driver plugins based on Network File System (NFS), CSI driver plugins based on distributed file system (Ceph), and CSI driver plugins based on local disks. For example, CSI driver plugin 1 can be an NFS-based CSI driver plugin, and CSI driver plugin 2 can be a local disk-based CSI driver plugin.

[0296] The CSI driver plugin can include volume mounters, container runtime components, and multiple mounters.

[0297] Volume mounters can be used to determine the type of container runtime components. A target node can include multiple types of container runtime components. For example, a target node can include runtime component 1 and runtime component 2, which are of different types.

[0298] Mounters can be used to perform operations such as creating volumes, mounting volumes, and unmounting volumes. Multiple mounters can include mounters corresponding to different runtime components. For example, multiple mounters can include mount 1 corresponding to runtime component 1 and mount 2 corresponding to runtime component 2.

[0299] In step 3.1, the node management component can send a volume mount request to the corresponding CSI driver plugin.

[0300] Specifically, the node management component can obtain the PVC information corresponding to the container running the target application through the volume manager. Then, based on the storage category identifier in the PVC information, it can determine the target storage category, and based on the CSI driver plugin identifier in the target storage category, it can determine the target CSI driver plugin corresponding to the PVC information among multiple CSI driver plugins.

[0301] The node management component can determine the target storage area information bound to the PVC information, as well as the target directory where the image file is to be mounted in the container running the target application. It can then generate a volume mount request and send the volume mount request to the target CSI driver plugin.

[0302] For example, the node management component can use the volume manager to determine the PVC information 1 corresponding to the container C1 running the target application, and determine that the target CSI driver plugin corresponding to PVC information 1 is CSI driver plugin 2. The node management component can generate a volume mount request and send the volume mount request to CSI driver plugin 2. The volume mount request can include the target address as address A, the target directory as " / data directory", and the identifier BD2 of the target storage region.

[0303] In step 3.2, after the target CSI driver plugin receives the volume mount request, it can call the volume mounter to determine the type of container runtime component in the target node, and determine the corresponding mounter according to the type of container runtime component, so as to call the corresponding mounter to perform the volume mount operation to mount the image file to the container. That is, the corresponding mounter can be called to execute steps 3.3 to 3.4.

[0304] For example, if CSI driver plugin 2 receives a volume mount request and calls the volume mounter to determine that the container runtime component in the target node is runtime component 1, then the corresponding mounter can be determined to be mounter 2.

[0305] In this application, the container runtime is dynamically detected by the volume mounter in the CSI driver plugin to determine the mounter that matches the container runtime. This enables adaptation to different container runtime environments and simplifies the unified management of data mounting in environments using different container runtime components.

[0306] In step 3.3, the mounter can determine the target storage region where the image file is located based on the identifier of the target storage region in the volume mount request. For example, if the volume mount request includes the identifier BD2 of the target storage region, the mounter 2 can determine that the target storage region is block device 2 based on the identifier BD2.

[0307] In step 3.4, the mounter can mount the image file as a data volume into the container used to run the target application by mounting the volume according to the target storage area.

[0308] It should be noted that the execution process of step 3.4 can be found in [reference needed]. Figure 3 Steps S303 to S310 in the embodiment will not be described again here.

[0309] By using the CSI driver plugin to mount the image file as a data volume into the container running the target application, the startup time of the Pod with this volume is in the second range, regardless of whether it is a single Pod startup or a multi-replica scaling scenario. Furthermore, the file contents of the image file can be read as local files. Therefore, the technical solution in this application can achieve the effect of mounting and reading large files or file groups in seconds.

[0310] Figure 8 A comparison chart showing the effects of this solution compared to other solutions is provided for this application. Please refer to [link / reference]. Figure 8 When loading 100M, 1GB, 5GB, 10GB and 30GB files, the loading time of this solution is less than that of other solutions. In particular, the optimization effect on loading time is more obvious when the file is larger.

[0311] By mounting image files as data volumes to containers used to run target applications, the traditional data management approach is changed. Image files are treated as part of the container, and image files are managed through the lifecycle management of container images, simplifying the distribution and update process of image files.

[0312] In this embodiment, the process includes generating an image file, pulling and saving the image file, and mounting the image file. During image file generation, at least one associated file with a large data volume can be packaged into an image file; during image file pulling and saving, the image file can be stored in the target storage area; during image file mounting, the image file can be mounted as a data volume into the container running the target application via volume mounting, eliminating the need to copy the image file. This achieves preloading of at least one associated file, reduces network bandwidth usage, accelerates the application startup process, and thus improves the efficiency of starting applications in containers that rely on image files.

[0313] Figure 9 A schematic diagram of a file processing system provided for an exemplary embodiment of this application. Please refer to... Figure 9 The system may include a CSI driver plugin, an image pre-puller, a node management component, and a CSI provisioner.

[0314] The CSI driver plugin can be used to execute... Figure 2 or Figure 3 The method shown in the example; the mirror prefetcher can be used to perform Figure 4 The method shown in the embodiment; the node management component can be used to execute Figure 5 The method shown in the embodiments; the CSI prefitter can be used to perform Figure 6 The method shown in the embodiment.

[0315] In this embodiment, since at least one associated file with a large amount of data can be packaged to generate an image file, the image pre-fetcher can pull the image file to the target storage area. The node management component sends a volume mount request to the CSI driver plugin, so that the CSI driver plugin determines the target storage area corresponding to the image file according to the volume mount request, and mounts the image file as a data volume to the container running the target application through volume mounting. It does not require copying the image file, realizing the preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting the application in the container that depends on the image file.

[0316] Figure 10 This is a schematic diagram of the cluster structure provided for an exemplary embodiment of this application. Please refer to... Figure 10 The cluster may include a first node, at least one second node, and a third node.

[0317] The first node contains an image prefetcher, which can be used for execution. Figure 4 The method shown in the embodiment.

[0318] For any given second node, a node management component and a CSI driver plugin can be configured, along with a corresponding target storage area for storing image files. The node management component can be used to execute... Figure 5 The method shown in the embodiment allows the CSI driver plugin to be used for execution. Figure 2 or Figure 3 The method shown in the embodiment.

[0319] For example, at least one second node can be second node 1, second node 2, ... Second node 1 can be equipped with a node management component and a CSI driver plugin, and has a corresponding target storage area, where image files can be stored.

[0320] The third node can contain a CSI configurer, which can be used for execution. Figure 6 The method shown.

[0321] In this embodiment, since at least one associated file can be decoupled from at least one application file, at least one associated file with a large amount of data can be packaged into an image file. The image pre-fetcher can pull the image file to the target storage area. The node management component sends a volume mount request to the CSI driver plugin so that the CSI driver plugin can determine the target storage area corresponding to the image file according to the volume mount request, and mount the image file as a data volume to the container running the target application through volume mount. There is no need to copy the image file, which realizes the preloading of at least one associated file, thereby accelerating the application startup process and improving the efficiency of starting the application in the container that depends on the image file.

[0322] Figure 11 This is a schematic diagram of a document processing apparatus provided in an embodiment of this application. Please refer to [link / reference]. Figure 11 The file processing device 10 may include: a determining module 11 and a mounting module 12, wherein,

[0323] The determining module 11 is used to determine the target storage area where the image file is located, wherein the image file is a file obtained by packaging at least one associated file that the target application depends on at runtime;

[0324] The mounting module 12 is used to mount the image file to a container for running the target application, based on the target storage area.

[0325] The document processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0326] In one possible implementation, the container runs on a target node in the cluster, and the target storage area is a block device mounted on the target node;

[0327] The image file is retrieved from the image repository and then pulled to the block device via an image pre-puller after being uploaded to the image repository by the user.

[0328] In one possible implementation, the image file is specifically pulled to the block device in the following manner:

[0329] After establishing the connection between the image repository and the image pre-puller, the image file is pulled from the image repository through the image pre-puller, and it is determined whether there is a mounted block device in the target node;

[0330] If it exists, the pulled image file will be stored in the mounted block device;

[0331] If it does not exist, the block device management component is invoked to create a new block device, and the newly created block device is mounted on the target node. The image file is then saved in the newly created block device.

[0332] In one possible implementation, the image file is obtained in the following manner:

[0333] Add the associated files that the target application depends on at runtime, as well as the corresponding auxiliary files, to the preset directory;

[0334] A manifest file is generated in the preset directory. The manifest file includes the hash values ​​corresponding to the associated files and auxiliary files, as well as the metadata corresponding to the associated files.

[0335] The system packages the files in the preset directory, calculates the hash value of the packaged files, and generates the corresponding image file based on the packaged files and the calculated hash value.

[0336] In one possible implementation, the file processing method is applied to the Container Storage Interface (CSI) driver plugin in the target node; the determining module 11 is specifically used for:

[0337] Get the volume mount request sent by the node management component;

[0338] Based on the volume mount request, determine the target storage area where the image file is located.

[0339] In one possible implementation, the target node includes multiple CSI driver plugins;

[0340] The target CSI driver plugin is determined by the node management component from among the plurality of CSI driver plugins based on the persistent volume declaration PVC information;

[0341] The persistent volume declaration information is generated by the CSI provisioner based on the persistent volume declaration request after it receives the user's input request for configuring associated files for the target application.

[0342] In one possible implementation, mounting the image file to a container for running the target application, based on the target storage region, includes:

[0343] Based on the target storage area, the image file is mounted as a data volume to the container used to run the target application via volume mounting.

[0344] In one possible implementation, the volume mount request includes the target address of the image file in the image repository and the target directory of the image file to be mounted in the container; the mount module 12 is specifically used for:

[0345] Determine whether the target address exists in the target storage area;

[0346] If it exists, then the target storage area is mounted to the target directory.

[0347] In one possible implementation, the mounting module 12 is further configured to:

[0348] If the target address does not exist in the target storage area, the target address is created in the target storage area, and the image file is pulled from the image repository to the target storage area according to the target address;

[0349] After the fetch is complete, the target storage area is mounted to the target directory.

[0350] In one possible implementation, if the volume mount request includes an update instruction, then before mounting the target storage area to the target directory, the mount module 12 is further configured to:

[0351] Determine whether the image file corresponding to the target address in the target storage area is consistent with the image file corresponding to the target address in the image repository;

[0352] If they match, a message indicating that an image file already exists will be returned.

[0353] If they are inconsistent, the image file corresponding to the target address is pulled from the image repository, and the image file in the target storage area is updated to the pulled image file.

[0354] In one possible implementation, the volume mount request includes a replacement strategy, and the mount module 12 is specifically used for:

[0355] If other data volumes are mounted under the target directory, then according to the replacement strategy, it is determined whether to replace the other data volumes with the target storage area;

[0356] If so, the target storage area is mounted to the target directory.

[0357] In one possible implementation, the file processing method is applied to a CSI driver plugin in a target node. The CSI driver plugin includes multiple mounters, each corresponding to a different runtime component in the target node. The mounting module 12 is specifically used for:

[0358] Determine the type of the container runtime component;

[0359] Depending on the type of the container runtime component, the corresponding mounter is invoked to perform a volume mount operation, thereby mounting the image file to the container.

[0360] In one possible implementation, the target application is used to run in multiple containers, with the same target storage region mounted to the multiple containers; or...

[0361] The target application shares the image file with other applications in the cluster, and the target storage area storing the image file is mounted to the container used to run the target application and the container used to run other applications.

[0362] Figure 12 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Please refer to... Figure 12 The electronic device 20 may include a processor 21 and a memory 22. Exemplarily, the processor 21 and the memory 22 are interconnected via a bus 23.

[0363] The memory 22 stores computer-executed instructions;

[0364] The processor 21 executes the computer execution instructions stored in the memory 22, causing the processor 21 to perform the method as shown in the above method embodiment.

[0365] Figure 12 The electronic device shown can be a node as described in any of the above embodiments.

[0366] Accordingly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the above-described method embodiments.

[0367] Accordingly, embodiments of this application may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the methods shown in the above-described method embodiments.

[0368] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0369] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0370] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0371] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0372] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0373] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0374] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0375] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0376] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A file processing method, characterized in that, The method includes: Determine the target storage area where the image file is located, wherein the image file is a file obtained by packaging at least one associated file that the target application depends on at runtime; Based on the target storage area, the image file is mounted to the container used to run the target application.

2. The method according to claim 1, characterized in that, The container runs on the target node in the cluster, and the target storage area is a block device mounted on the target node; The image file is retrieved from the image repository and then pulled to the block device via an image pre-puller after being uploaded to the image repository by the user.

3. The method according to claim 2, characterized in that, The image file is specifically pulled to the block device in the following manner: After establishing the connection between the image repository and the image pre-puller, the image file is pulled from the image repository through the image pre-puller, and it is determined whether there is a mounted block device in the target node; If it exists, the pulled image file will be stored in the mounted block device; If it does not exist, the block device management component is invoked to create a new block device, and the newly created block device is mounted on the target node. The image file is then saved in the newly created block device.

4. The method according to claim 2, characterized in that, The image file was obtained in the following way: Add the associated files that the target application depends on at runtime, as well as the corresponding auxiliary files, to the preset directory; A manifest file is generated in the preset directory. The manifest file includes the hash values ​​corresponding to the associated file and the auxiliary file, as well as the metadata corresponding to the associated file. The system packages the files in the preset directory, calculates the hash value of the packaged files, and generates the corresponding image file based on the packaged files and the calculated hash value.

5. The method according to any one of claims 1-4, characterized in that, The file processing method is applied to the container storage interface (CSI) driver plugin in the target node. Determining the target storage area where the image file is located includes: Obtain the volume mount request sent by the node management component; Based on the volume mount request, determine the target storage area where the image file is located.

6. The method according to claim 5, characterized in that, The target node includes multiple CSI driver plugins; The target CSI driver plugin is determined by the node management component from among the plurality of CSI driver plugins based on the persistent volume declaration PVC information; The persistent volume declaration information is generated by the CSI provisioner based on the persistent volume declaration request after it receives the user's input request for configuring associated files for the target application.

7. The method according to any one of claims 1-6, characterized in that, Mounting the image file to a container for running the target application, based on the target storage region, includes: Based on the target storage area, the image file is mounted as a data volume to the container used to run the target application via volume mounting.

8. The method according to claim 7, characterized in that, The volume mount request includes the target address of the image file in the image repository, and the target directory in the container where the image file is to be mounted; Based on the target storage area, the image file is mounted as a data volume to the container used to run the target application via volume mounting, including: Determine whether the target address exists in the target storage area; If it exists, then the target storage area is mounted to the target directory.

9. The method according to claim 8, characterized in that, The method further includes: If the target address does not exist in the target storage area, the target address is created in the target storage area, and the image file is pulled from the image repository to the target storage area according to the target address; After the fetch is complete, the target storage area is mounted to the target directory.

10. The method according to claim 8, characterized in that, If the volume mount request includes an update instruction, the method further includes, before mounting the target storage region to the target directory: Determine whether the image file corresponding to the target address in the target storage area is consistent with the image file corresponding to the target address in the image repository; If they match, a message indicating that an image file already exists will be returned. If they are inconsistent, the image file corresponding to the target address is pulled from the image repository, and the image file in the target storage area is updated to the pulled image file.

11. The method according to claim 8, characterized in that, The volume mount request includes a replacement strategy to mount the target storage area to the target directory, including: If other data volumes are mounted under the target directory, then according to the replacement strategy, it is determined whether to replace the other data volumes with the target storage area; If so, the target storage area is mounted to the target directory.

12. The method according to claim 7, characterized in that, The file processing method is applied to the CSI driver plugin in the target node. The CSI driver plugin includes multiple mounters, and each of the multiple mounters corresponds to a multiple runtime component in the target node. Mounting the image file as a data volume to the container used to run the target application involves: Determine the type of the container runtime component; Depending on the type of the container runtime component, the corresponding mounter is invoked to perform a volume mount operation, thereby mounting the image file to the container.

13. The method according to any one of claims 1-4, characterized in that, The target application is intended to run in multiple containers, and the same target storage region is mounted to the multiple containers; or... The target application shares the image file with other applications in the cluster, and the target storage area storing the image file is mounted to the container used to run the target application and the container used to run other applications.

14. A file processing method, characterized in that, Applied to a mirror prefetcher, the method includes: Pull image files from the image repository, wherein the image file is a file obtained by packaging at least one associated file that the target application depends on at runtime; The image file is stored in the target storage area and is used to mount the container for running the target application.

15. A document processing method, characterized in that, Applied to a node management component, the method includes: Retrieve persistent volume declaration information; Based on the persistent volume declaration information, a volume mount request is determined and sent to the corresponding CSI driver plugin. The volume mount request is used to instruct the CSI driver plugin to determine the target storage area corresponding to the image file, and to mount the image file to the container used to run the target application based on the target storage area. The image file is a file obtained by packaging at least one associated file that the target application depends on at runtime.

16. A file processing method, characterized in that, Applied to CSI prefitters, the method includes: Obtain the persistent volume declaration request from the user, which is used to configure associated files for the target application; Based on the persistent volume declaration request, a corresponding volume creation request is determined and sent to the CSI driver plugin. The volume creation request is used to instruct the CSI driver plugin to determine the target storage area corresponding to the image file and call the image pre-puller to store the image file in the target storage area. The image file is a file obtained by packaging at least one associated file that the target application depends on at runtime. The image file is used to mount to the container used to run the target application.

17. A document processing system, characterized in that, include: CSI driver plugin, image pre-puller, node management component, and CSI provisioner; The CSI driver plugin is used to perform the method described in any one of claims 1-13; The mirror pre-puller is used to perform the method of claim 14; The node management component is used to perform the method of claim 15; The CSI prefitter is used to perform the method of claim 16.

18. A cluster, characterized in that, It includes a first node, at least one second node, and a third node. The first node is equipped with an image pre-puller. The second node is equipped with a node management component and a CSI driver plugin, and has a corresponding target storage area for storing image files. The third node is equipped with a CSI provisioner. The CSI driver plugin is used to perform the method described in any one of claims 1-13; The mirror pre-puller is used to perform the method of claim 14; The node management component is used to perform the method of claim 15; The CSI prefitter is used to perform the method of claim 16.

19. A document processing device, characterized in that, The device includes: a determining module and a mounting module, wherein... The determining module is used to determine the target storage area where the image file is located, wherein the image file is a file obtained by packaging at least one associated file that the target application depends on at runtime; The mounting module is used to mount the image file to a container for running the target application, based on the target storage area.

20. An electronic device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1-16.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-16.

22. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-16.

Images