Data access method, physical machine, computer cluster and program product

Abstract

The invention provides a data access method, a physical machine, a computer cluster and a program product. A cache manager and a virtual machine instance are deployed on the physical machine, and the virtual machine instance comprises a cache file system, a control device and a corresponding shared device. The control device sends a second access request carrying the name identifier of the target resource to the cache manager in response to the first access request of the cache file system; the cache manager obtains an index identifier corresponding to the target resource based on the second access request, and sends the index identifier to the control equipment; and the control equipment maps the index identifier to the corresponding sharing equipment, and returns a virtual address space of the target resource in the sharing equipment to the cache file system, so that the cache file system accesses the target resource based on the virtual address space. According to the method, the security of data access of the virtual machine instance to the physical machine can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and in particular, to a data access method, a physical machine, a computer cluster, and a program product. Background Art

[0002] In a virtualized environment, a physical machine (Host) is a physical machine that runs virtualization software and is responsible for managing and allocating physical hardware resources to multiple virtual machine instances (Guest). Generally, a virtual machine instance cannot directly access the file system of the physical machine.

[0003] Currently, if a virtual machine instance needs to access the file system of a physical machine, it needs to perform indirect data access to the file system of the physical machine based on a file system sharing technology. A commonly used file system sharing technology is Virtio-fs (the name of a technology for achieving efficient file system sharing in a virtualized environment).

[0004] However, when a virtual machine instance uses the Virtio-fs technology to access the file system of a physical machine, it needs to map a complete file system directory structure specific to the physical machine into the shared memory space of the virtual machine instance, which may lead to a problem of relatively low security of this data access method. Summary of the Invention

[0005] This application provides a data access method, a physical machine, a computer cluster, and a program product, which can improve the security of data access by a virtual machine instance to a physical machine.

[0006] In a first aspect, this application provides a data access method. A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device. The method includes:

[0007] In response to a first access request for a target resource, the control device sends a second access request to the cache manager. The first access request is sent by the cache file system to the control device. The second access request carries: a name identifier of the target resource;

[0008] The control device receives a first access response returned by the cache manager based on the second access request. The first access response carries: an index identifier corresponding to the target resource;

[0009] The control device maps the index identifier to the corresponding shared device and returns a second access response to the cache file system. The second access response includes: a virtual address space of the target resource in the shared device, so that the cache file system can access the target resource through the virtual address space.

[0010] Optionally, different control devices correspond to different data sources, and the second access request further carries: an identifier of the target data source corresponding to the control device.

[0011] Optionally, the first access request carries: a name identifier of the target resource. In response to the first access request for the target resource, the control device sends a second access request to the cache manager, including:

[0012] The control device performs a security check on the first access request;

[0013] When the security check of the first access request passes, the control device adds the identifier of the target data source to the first access request to obtain the second access request;

[0014] The control device sends the second access request to the cache manager.

[0015] Optionally, the first access request is a read request. After the control device sends a second access request to the cache manager in response to the first access request for the target resource, the method further includes:

[0016] When the control device fails to obtain the index identifier from the cache manager within a preset duration, the control device sends a third access response to the cache file system, and the third access response is used to indicate that the target resource is not cached on the physical machine.

[0017] Optionally, before the control device sends a second access request to the cache manager in response to the first access request for the target resource, the method further includes:

[0018] The control device establishes a connection with the cache file system through a corresponding virtual port.

[0019] In a second aspect, the present application provides a data access method. A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device. The method includes:

[0020] The shared device receives a data access request sent by the cache file system. The data access request includes: a virtual address space of the target resource in the shared device; the virtual address space is obtained based on the method according to any item in the first aspect.

[0021] In response to the data access request, the shared device feeds back the target resource to the cache file system.

[0022] In a third aspect, the present application provides a data access method. A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device. The method includes:

[0023] The cache file system sends a first access request for a target resource to the control device, so that in response to the first access request, the control device sends a second access request to the cache manager, and after receiving a first access response returned by the cache manager based on the second access request, maps an index identifier corresponding to the target resource to the corresponding shared device, and returns a second access response to the cache file system; the second access request carries: a name identifier of the target resource; the first access response carries: the index identifier; the second access response includes: a virtual address space of the target resource in the shared device;

[0024] The cache file system receives the second access response, and based on the virtual address space, accesses the target resource through a direct access DAX channel between the cache file system and the shared device.

[0025] Optionally, before the cache file system sends a first access request for a target resource to the control device, the method further includes:

[0026] The cache file system listens to a virtual port corresponding to the control device to determine whether the control device establishes a connection through the virtual port;

[0027] The cache file system sending a first access request for a target resource to the control device includes:

[0028] The cache file system sends the first access request to the control device when a connection is established with the control device.

[0029] Optionally, when the first access request is a read request, after the cache file system sends a first access request for a target resource to the control device, the method further includes:

[0030] The cache file system obtains the target resource from an object storage service system or a network file system in response to a third access response from the control device; the third access response is used to indicate that the physical machine does not cache the target resource; the third access response is sent by the control device to the cache file system in response to not obtaining the index identifier from the cache manager within a preset duration.

[0031] Optionally, after the cache file system obtains the target resource from the object storage service system or the network file system in response to a third access response from the control device, the method further includes:

[0032] The cache file system sends a third access request to the control device; the third access request is used to request to write the target resource into the cache manager.

[0033] In a fourth aspect, the present application provides a data access method. A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device. The method includes:

[0034] The cache manager receives a second access request sent by the control device. The second access request is sent by the control device in response to a first access request for a target resource, and the first access request is sent by the cache file system to the control device. The second access request carries: a name identifier of the target resource;

[0035] The cache manager obtains an index identifier corresponding to the target resource based on the name identifier of the target resource;

[0036] The cache manager sends a first access response carrying the index identifier to the control device, so that the control device maps the index identifier to the corresponding shared device and returns a second access response to the cache file system. The second access response includes: a virtual address space of the target resource in the shared device, so that the cache file system accesses the target resource through the virtual address space.

[0037] Optionally, the first access request is a read request, and the method further includes:

[0038] When the cache manager fails to query the index identifier corresponding to the target resource based on the name identifier of the target resource, the cache manager sends a third access response to the cache file system through the control device. The third access response is used to indicate that the physical machine does not cache the target resource.

[0039] In a fifth aspect, the present application provides a data access method. A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device. The method includes:

[0040] The cache file system sends a first access request for a target resource to the control device;

[0041] In response to the first access request, the control device sends a second access request to the cache manager; the second access request carries: the name identifier of the target resource;

[0042] Based on the name identifier of the target resource, the cache manager obtains the index identifier corresponding to the target resource;

[0043] The cache manager sends a first access response to the control device, and the first access response carries: the index identifier;

[0044] The control device maps the index identifier to the corresponding shared device and returns a second access response to the cache file system; the second access response includes: the virtual address space of the target resource in the shared device;

[0045] In response to the second access response, the cache file system accesses the target resource based on the virtual address space through the direct access DAX channel with the shared device.

[0046] In a sixth aspect, the present application provides a physical machine, on which a cache manager and at least one virtual machine instance are deployed. The virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device, and different control devices correspond to different data sources;

[0047] Wherein, the control device is used to execute the method according to any one of the first aspect;

[0048] The shared device is used to execute the method according to the second aspect;

[0049] The cache file system is used to execute the method according to any one of the third aspect;

[0050] The cache manager is used to execute the method according to any one of the fourth aspect.

[0051] In a seventh aspect, the present application provides a computer cluster, which includes at least one physical machine according to the sixth aspect.

[0052] In an eighth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of the first aspect, the second aspect, the third aspect, and the fourth aspect.

[0053] In a ninth aspect, the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the method described in any one of the first aspect, the second aspect, the third aspect, and the fourth aspect.

[0054] The data access method, physical machine, computer cluster, and program product provided by the present application add a control device to a virtual machine instance. The control device can obtain a first access request sent by the cache file system of the virtual machine instance, and based on this first access request, obtain an index identifier of a target resource from a cache manager, laying a foundation for mapping the index identifier to a shared device to obtain the virtual address space of the target resource. By sending a second access response including the virtual address space of the target resource to the cache file system, the cache file system can access the target resource based on the virtual address space of the target resource, that is, while realizing the access to the target resource, it is not necessary to map the complete file system directory structure to the shared memory space, thus improving the security of data access. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 A structural schematic diagram of a physical machine provided by an embodiment of the present application;

[0057] Figure 2 A flowchart of a data access method provided by an embodiment of the present application;

[0058] Figure 3 Another structural schematic diagram of a physical machine provided by the present application;

[0059] Figure 4 A hardware structural schematic diagram of an electronic device provided by the present application.

[0060] Through the above drawings, the clear embodiments of the present application have been shown, and there will be more detailed descriptions later. These drawings and the text description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following will clearly and completely describe the technical solutions in this application in conjunction with the accompanying drawings in this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.

[0062] The following explains the terms related to this application:

[0063] Virtualization layer: A technical layer located between the physical machine and virtual machine instances, responsible for implementing virtualization functions. The virtualization layer provides an abstraction layer between the physical machine and virtual machine instances, enabling virtual machine instances to run without knowing the specific implementation of the underlying hardware, and realizing the sharing, isolation, and dynamic allocation of hardware resources.

[0064] Direct Access (DAX): A mechanism that enables user-level applications of virtual machine instances to directly access data in the physical machine's file system. It allows user-level applications of virtual machine instances to directly access the mapped files of persistent memory without first copying the file system data to the page cache.

[0065] Filesystem in Userspace (FUSE): A framework that allows user-level applications to create and manage file systems. It provides a mechanism that enables non-privileged user-level applications to create fully functional file systems without recompiling the kernel.

[0066] In a virtualized environment, the physical machine is the physical machine running virtualization software, responsible for managing and allocating physical hardware resources to multiple virtual machine instances. Virtual machine instances run as independent operating system instances on the physical machine and interact with the physical hardware through the virtualization layer. Each virtual machine instance has its own operating system kernel, user space, and resource allocation, and is isolated from the physical machine and other virtual machine instances, thus ensuring that each virtual machine instance can independently run applications and services.

[0067] Since virtual machine instances are isolated from the physical machine, virtual machine instances cannot directly access the physical machine's file system. Therefore, it is necessary to set up a shared memory area between the physical machine and virtual machine instances to achieve data transmission between the physical machine and virtual machine instances through data transmission technologies.

[0068] The shared memory area is a physical memory space located on the physical machine, which is mapped to the address space of the virtual machine instance through virtualization technology, enabling the virtual machine instance to directly access this memory.

[0069] Zero-copy data transfer is a data transfer technology that significantly improves data transfer efficiency by reducing unnecessary data copy operations and directly transferring data between virtual machine instances and physical machines.

[0070] Currently, zero-copy data transfer between physical machines and virtual machine instances can be achieved through file system sharing technologies. A commonly used file system sharing technology is the Virtio-fs technology. Among them, Virtio-fs is a technology for implementing file system sharing in a virtualized environment. The core components of this technology include the FUSE daemon on the physical machine, the kernel FUSE module, and the Virtio-fs device on the virtual machine instance side. The above components work together to achieve file system data transfer between the virtual machine instance and the physical machine.

[0071] The Virtio-fs device driver in the Virtio-fs device is responsible for transferring file system requests and responses between the virtual machine instance and the physical machine. The Virtio-fs device establishes an efficient communication channel between the virtual machine instance and the physical machine through the Virtio protocol, and uses shared memory areas and DAX functions to achieve zero-copy data transfer, enabling the virtual machine instance to access data resources on the physical machine as efficiently as accessing the local file system.

[0072] During the implementation process, the Virtio-fs device maps the file system structure and content on the physical machine completely into the virtual machine instance. Therefore, the file directory structure, file size, and content seen by the virtual machine instance are the same as those of the physical machine. This feature enables the virtual machine instance to bypass its own page cache and directly access the data on the physical machine through the DAX function, further improving data transfer efficiency.

[0073] However, the method of mapping a complete file system directory structure into the shared memory space of the virtual machine instance, allowing the virtual machine instance to directly access files on the physical machine, has certain security risks and may lead to relatively low data access security.

[0074] Considering the above problems existing in the existing data access methods, therefore, the embodiments of this application propose a data access method that does not require mapping a complete file system directory structure into the shared memory space to improve data access security.

[0075] The technical solutions of this application will be described in detail below in combination with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0076] Figure 1A schematic structural diagram of a physical machine provided by an embodiment of the present application. As Figure 1 shown, the following can be deployed on the physical machine 10: a cache manager 11, and at least one virtual machine instance 12. Among them, the virtual machine instance 12 may include: a cache file system 13, a control device 14, and a shared device 15 corresponding to the control device 14.

[0077] Exemplarily, the above cache manager 11 can be used to manage and maintain the cache content in the persistent memory of the physical machine 10, and transfer and update the cache data. Optionally, the cache manager 11 can be, for example, a background management process. In some embodiments, the cache manager 11 can be, for example, a cache daemon for the cache file system 13 (Cache FileSystem cache daemon, cachefs cache daemon). Among them, the above persistent memory can be, for example, a memory cache (Memory Cache), or a disk cache (Disk Cache), etc. It should be understood that the specific form of the cache manager 11 in the embodiments of the present application is not limited.

[0078] Taking the example that multiple virtual machine instances 12 are deployed on the physical machine 10, the multiple virtual machine instances 12 may all include a cache file system 13, a control device 14, and a shared device 15 corresponding to the control device 14, and the above components can be used to execute the data access method provided by any embodiment of the present application. Or, among the multiple virtual machine instances 12, there may be some virtual machine instances 12 that can execute the data access method provided by any embodiment of the present application. It should be understood that the number of virtual machine instances 12 deployed on the physical machine 10 in the present application is not limited.

[0079] In some embodiments, the above cache file system 13 can refer to the cachefs fuse in any existing virtual machine instance 12. Exemplarily, the cache file system 13 can be used to accelerate the read-only data access efficiency of the virtual machine by means of local caching, etc.

[0080] Exemplarily, the above control device 14 and the shared device 15 corresponding to the control device 14 can be preset in the virtual machine instance 12 (for example, as a process of the virtual machine instance 12, preset in the virtual machine instance 12). In some embodiments, the control device 14 can also be referred to as a cachefs device.

[0081] Optionally, the cache file system 13 included in the virtual machine instance 12 may correspond to the control device 14. Optionally, for any virtual machine instance 12 deployed on the physical machine 10, taking the virtual machine instance 12 including one control device 14 as an example, the virtual machine instance 12 may include a cache file system 13 corresponding to the control device 14. Alternatively, in some embodiments, taking the virtual machine instance 12 including multiple control devices 14 as an example, the virtual machine instance 12 may include multiple cache file systems 13, and the multiple cache file systems 13 may correspond to the multiple control devices 14 one by one.

[0082] Exemplarily, the above-mentioned shared device 15 may refer to, for example, any existing shared device 15 in the virtual machine instance 12. For example, the shared device 15 may be the aforementioned Virtio-fs device.

[0083] Based on the above physical machine 10, the present application provides a data access method. Figure 2 It is a schematic flowchart of a data access method provided by an embodiment of the present application. As Figure 2 shown, the method may include the following steps:

[0084] S101. The cache file system 13 sends a first access request for a target resource to the control device 14.

[0085] Exemplarily, the above-mentioned target resource may be data in any form such as a file, tabular data, etc. The present application does not limit the data type of the target resource.

[0086] Optionally, the above first access request may be a read request or a write request. Taking the first access request as a read request as an example, the first access request may be used to request to read the above target resource. Taking the first access request as a write request as an example, the first access request may be used to request to write the above target resource to the physical machine 10, for example.

[0087] Exemplarily, the above first access request may include, for example, the name identifier of the target resource. Exemplarily, taking the target resource as a file as an example, the name identifier of the target resource may be the name of the file. Or, taking the target resource as tabular data as an example, the name of the target resource may be information such as the table header of the table. Or, the name of the target resource may also be the label of the data block, etc.

[0088] Exemplarily, the above first access request may also be used to describe resource requirements. The resource required by the resource requirements is the above target resource. Or, in some embodiments, the above first access request may also refer to the content of the access request sent by any existing cache file system 13 to access data on the physical machine 10, which will not be elaborated herein.

[0089] It should be understood that this application does not limit the timing of the above cache file system 13 sending the first access request for the target resource to the control device 14. For example, the cache file system 13 may send the first access request for the target resource to the control device 14 in response to a target operation of the user on the virtual machine instance 12. Among them, the above target operation may be an operation that needs to be executed based on the target resource. Or, for another example, the cache file system 13 may also send the first access request for the target resource to the control device 14 in response to an acquisition instruction of the target resource triggered during the process running of the virtual machine instance 12.

[0090] Exemplarily, taking the above cache file system 13 and the control device 14 as different processes of the virtual machine instance 12, the process of the cache file system 13 may generate the above first access request, and synchronize the first access request to the process of the control device 14, so as to send the first access request to the control device 14.

[0091] Correspondingly, the control device 14 may receive the first access request.

[0092] S102. In response to the above first access request for the target resource, the control device 14 sends a second access request to the cache manager 11.

[0093] Among them, the second access request may carry: the name identifier of the above target resource.

[0094] As a possible implementation manner, for example, after receiving the above first access request, the control device 14 may use the first access request as the second access request. As another possible implementation manner, taking the above first access request including description information (such as text) for describing resource requirements as an example, the control device 14 may first determine the target resource required by the resource requirement based on the description information. Then, the control device 14 generates the above second access request based on the name identifier of the target resource.

[0095] Optionally, the implementation manner of the control device 14 sending the second access request to the cache manager 11 may refer to any existing interaction method between the virtual machine instance 12 and the cache manager 11 of the physical machine 10, which will not be elaborated here.

[0096] Correspondingly, the cache manager 11 may receive the second access request sent by the control device 14.

[0097] S103. The cache manager 11 obtains the index identifier corresponding to the target resource based on the name identifier of the target resource.

[0098] Exemplarily, the index identifier corresponding to the above target resource can be, for example, the file descriptor (FD) corresponding to the target resource. The FD can be a non - negative integer index. Taking the index identifier as FD as an example, optionally, the cache manager 11 obtains the implementation method of the FD corresponding to the target resource based on the name identifier of the target resource. For example, it can refer to the implementation method of obtaining the FD of the resource to be accessed based on the name identifier of the resource to be accessed in any existing data access method. This application will not elaborate here. S104. The cache manager 11 sends a first access response to the control device 14.

[0099] Among them, the first access response can carry: the above - mentioned index identifier.

[0100] Optionally, after determining the above - mentioned index identifier, the cache manager 11 can generate a first access response including the index identifier based on the index identifier.

[0101] Correspondingly, the control device 14 can receive the first access response.

[0102] S105. The control device 14 maps the index identifier to the corresponding shared device 15 and determines the virtual address space of the target resource in the shared device 15.

[0103] Exemplarily, the above - mentioned virtual address space can be represented by, for example, an offset and a data volume size. Among them, the offset can be, for example, the offset of the storage location of the target resource compared to a specific location. Exemplarily, the specific location can be, for example, the start location, etc. Or, the representation method of the virtual address space (or the content included in the virtual address space) can refer to any existing address used when accessing data based on the shared device 15. This application does not limit this.

[0104] Optionally, taking the shared device 15 as a Virtio - fs device as an example, the control device 14 can map the index identifier to the corresponding shared device 15 through the Virtio protocol to obtain the virtual address space of the target resource in the shared device 15.

[0105] S106. The control device 14 returns a second access response to the cache file system 13.

[0106] Correspondingly, the cache file system 13 can receive the second access response.

[0107] Among them, the second access response may include: the virtual address space of the above-mentioned target resource in the shared device 15, so that the cache file system 13 can access the above-mentioned target resource through the virtual address space. Exemplarily, the cache file system 13 may, for example, send an access request including the virtual address space to the above-mentioned shared device 15 to access the above-mentioned target resource through the shared device 15.

[0108] In this embodiment, by adding the control device 14 in the virtual machine instance 12, the control device 14 can obtain the first access request sent by the cache file system 13 of the virtual machine instance 12, and based on the first access request, obtain the index identifier of the target resource from the cache manager 11, laying a foundation for mapping the index identifier to the shared device 15 to obtain the virtual address space of the target resource. By sending the second access response including the virtual address space of the target resource to the cache file system 13, the cache file system 13 can access the target resource based on the virtual address space of the target resource, that is, while realizing the access to the target resource, it is not necessary to map the complete file system directory structure to the shared memory space, thus improving the security of data access.

[0109] In some embodiments, different control devices 14 may correspond to different data sources. Optionally, taking the example that multiple virtual machine instances 12 are deployed on the physical machine 10, the different control devices 14 may refer to the control devices 14 included in different virtual machine instances 12. Taking the example that the virtual machine instance 12 includes multiple control devices 14, the above-mentioned different control devices 14 may also refer to different control devices 14 included in the same virtual machine instance 12.

[0110] Optionally, the above-mentioned data source may refer to the data source stored in the physical machine 10. It should be understood that the present application does not limit the form of the data source. For example, the data source may be: any existing data source in the physical machine 10 such as a file, a folder, data, or a table.

[0111] The above-mentioned second access request may also carry: the identifier of the target data source corresponding to the control device 14.

[0112] As mentioned above, different control devices 14 may correspond to different data sources. The target data source is the data source corresponding to the control device 14. Optionally, the identifier of the above-mentioned target data source may be used to represent the range of the data source that can be accessed through the control device 14. Exemplarily, the identifier of the above-mentioned target data source may be an identifier with global uniqueness. In some instances, the identifier of the above-mentioned target data source may also be referred to as the tag of the target data source.

[0113] For example, taking the target data source as an example of mounting an Object Storage Service (OSS), the identifier of the target data source can be, for example, the name of the bucket of the OSS (i.e., the namespace in the OSS, which is equivalent to a container for storing objects). The name of the bucket of the OSS is globally unique and serves as the identifier of the target data source.

[0114] Optionally, the identifier of the target data source corresponding to the control device 14 can be, for example, pre-configured in the control device 14.

[0115] Optionally, the cache manager 11 can obtain the index identifier corresponding to the target resource based on the identifier of the target data source and the name identifier of the target resource. For example, the cache manager 11 can first determine the range of data sources that the control device 14 can access based on the identifier of the target data source. For example, a mapping relationship between the identifier of the data source and the range of accessible data sources can be pre-stored in the cache manager 11. The cache manager 11 can determine the range of data sources that the control device 14 can access based on the identifier of the target data source and this mapping relationship. Exemplarily, the mapping relationship between the identifier of the data source and the range of accessible data sources can be as shown in Table 1 below:

[0116] Table 1

[0117]

[0118] Exemplarily, taking the identifier of the target data source as the identifier 3 of the data source as an example, based on the mapping relationship shown in Table 1 above, the control device 14 can determine that the range of data sources that the control device 14 can access is the range 3 of the data source.

[0119] After the cache manager 11 determines the range of data sources that the control device 14 can access, for example, it can search for the index identifier corresponding to the target resource within this range of data sources based on the name identifier of the target resource. If the cache manager 11 finds the index identifier corresponding to the target resource within this range of data sources, it can execute the following step S104 to send a first access response including this index identifier to the control device 14.

[0120] In some embodiments, taking the example that a data source range indication label (e.g., an identifier of a data source corresponding to the control device 14 that is allowed to access the target resource) is pre-added to the target resource in the physical machine 10, the cache manager 11 can, for example, first search for the target resource based on the name identifier of the target resource. Then, when the data source range indication label of the target resource is the same as the identifier of the target data source corresponding to the control device 14, the cache manager 11 can obtain the index identifier corresponding to the target resource.

[0121] In this embodiment, the control device 14 sends to the cache manager 11 an identifier of the target data source corresponding to the control device 14, and different control devices 14 can correspond to different data sources, enabling the cache management to isolate the ranges of data sources that different control devices 14 can access based on the identifiers of the data sources sent by different control devices 14, thereby improving data security in the scenario of multiple control devices 14.

[0122] The following details how the control device 14 responds to the first access request for the target resource and sends a second access request to the cache manager 11:

[0123] As a possible implementation, the control device 14 can, for example, first perform a security check on the first access request.

[0124] Exemplarily, the above security check can include, for example, verification check, and / or rate limiting check, etc.

[0125] Taking the security check including verification check as an example, exemplarily, the control device 14 can, for example, perform a verification check on the first access request based on the data transfer protocol with the cache file system 13, such as verifying aspects such as the format correctness and content integrity of the first access request. For example, the control device 14 can determine that the security check of the first access request passes after the verification of the format correctness and content integrity of the first access request is passed. If the first access request does not meet the verification checks such as format correctness or content integrity, the control device 14 can determine that the security check of the first access request fails.

[0126] Taking the security check including the flow limit check as an example, exemplarily, after receiving the above first access request, the control device 14 can, for example, add the first access request to the queue of requests to be processed. Then, the control device 14 can, for example, determine whether the flow limit check passes based on the number of access requests to be processed in the queue. The control device 14 can, for example, determine that the security check of the first access request fails when the number of access requests to be processed in the queue is greater than the preset number. The control device 14 can, for example, determine that the security check of the first access request passes when the number of access requests to be processed in the queue is less than or equal to the preset number.

[0127] Taking the example that the first access request carries the name identifier of the target resource, when the security check of the first access request passes, the control device 14 can add the identifier of the target data source to the first access request to obtain a second access request. Then, the control device 14 can send the second access request to the cache manager 11.

[0128] Exemplarily, taking the example that the first access request includes a blank identification bit, the control device 14 can, for example, add the identifier of the target data source to the blank identification bit of the first access request to obtain a second access request. Alternatively, the control device 14 can, for example, also use the tagging method to use the identifier of the target data source as a tag for the first access request. The control device 14 can use the first access request with the identifier of the target data source added as a tag as the second access request.

[0129] In this embodiment, the control device 14 can perform a security check on the first access request. When the security check of the first access request passes, it indicates that processing the first access request can ensure the security of the operation of the virtual machine instance 12 and the physical machine 10. Therefore, when the security check of the first access request passes, the identifier of the target data source is added to the first access request to obtain a second access request and send it to the cache manager 11, improving the security of data transmission between the control device 14 and the cache manager 11, and thus further improving the security of the data access process.

[0130] The following is an exemplary description of how the cache file system 13 accesses the target resource after receiving the above second access response:

[0131] As a possible implementation, the cache file system 13 can, for example, respond to the above second access response and access the target resource through the DAX channel with the shared device 15 based on the virtual address space carried by the second access response.

[0132] Exemplarily, taking the above-mentioned virtual address space including an offset and the size of the target resource as an example, the cache file system 13 can, for example, access the sparse file of the physical machine 10 based on the offset and the size of the target resource through the DAX channel with the shared device 15 to achieve access to the target resource. Exemplarily, the offset can be, for example, the offset compared to the position of sparse file 0. For example, if the offset is 2 Gigabytes (G) and the size of the target resource is 1G, the cache file system 13 can access 1G of data resources starting from the 2G data position of the sparse file as the target resource through the DAX channel with the shared device 15.

[0133] In some embodiments, the cache file system 13 can, for example, send a data access request to the shared device 15, requesting to access the target resource through the DAX channel with the shared device 15.

[0134] Correspondingly, the shared device 15 can receive the data access request sent by the cache file system 13. Among them, the above data access request can include: the virtual address space of the target resource in the shared device 15. This virtual address space is obtained based on the data access method described in any of the foregoing embodiments.

[0135] Then, the shared device 15 can respond to the data access request and feedback the target resource to the cache file system 13.

[0136] Optionally, the shared device 15 can, for example, respond to the data access request and establish a DAX channel between the shared device 15 and the cache file system 13, so that the cache file system 13 can access the target resource through the DAX channel with the shared device 15.

[0137] In this embodiment, the control device 14 enables the cache file system 13 to access the target resource based on the virtual address space through the DAX channel with the shared device 15 by sending a second access response including the virtual address space of the target resource to the cache file system 13, realizing high-efficiency zero-copy data transmission between the cache file system 13 and the physical machine 10 through DAX shared memory, that is, improving the efficiency of data transmission.

[0138] As a possible implementation, before the cache file system 13 sends a first access request for the target resource to the control device 14 (and also before the control device 14 responds to the first access request for the target resource and sends a second access request to the cache manager 11), for example, it can first establish a connection with the control device 14.

[0139] For example, the cache file system 13 can monitor the virtual port corresponding to the control device 14 to determine whether the control device 14 establishes a connection through the virtual port.

[0140] Exemplarily, the above cache file system 13 can, for example, monitor the virtual port of a virtual socket (vsock) through a process and wait for the control device 14 to connect.

[0141] The control device 14 can, for example, establish a connection with the cache file system 13 through the corresponding virtual port, so that when the cache file system 13 establishes a connection with the control device 14, the cache file system 13 sends the foregoing first access request to the control device 14.

[0142] Through the above method, by monitoring the virtual port corresponding to the control device 14, the cache file system 13 can send the first access request to the control device 14 in a timely manner after the control device 14 is connected to the cache file system 13, laying a foundation for subsequently obtaining the virtual address space of the target resource based on the control device 14.

[0143] As a possible implementation, taking the first access request as a read request as an example, when the cache manager 11 fails to query the index identifier corresponding to the target resource based on the name identifier of the target resource, the cache manager 11 can send a third access response to the cache file system 13 through the control device 14.

[0144] Wherein, the above third access response can be used to indicate that the physical machine does not cache the above target resource.

[0145] If the cache manager 11 fails to query the index identifier corresponding to the target resource based on the name identifier of the target resource, it indicates that the physical machine 10 does not store the target resource, or the target resource is not within the range of data sources that the control device 14 can access. Therefore, the cache manager 11 can send the above third access response to the control device 14.

[0146] Correspondingly, the control device 14 can receive the above third access response.

[0147] After the control device 14 sends a second access request to the cache manager 11 in response to the first access request for the target resource and fails to obtain the index identifier from the cache manager 11 within a preset time period, the control device 14 can send the above third access response to the cache file system 13.

[0148] Exemplarily, the preset time period can be pre-stored in the control device 14. Exemplarily, the preset time period can be, for example, 10 seconds or other time periods. The present application does not limit the preset time period.

[0149] Exemplarily, when receiving the above-mentioned third access response from the cache manager 11, the control device 14 can determine that the index identifier has not been obtained from the cache manager 11.

[0150] Correspondingly, the cache file system 13 can receive the above-mentioned third access response.

[0151] Through the above method, when the control device 14 fails to obtain the above-mentioned index identifier from the cache manager 11, it can send a third access response to the cache file system 13, enabling the cache file system 13 to promptly determine that it cannot access the target resource from the cache manager 11, laying a foundation for the subsequent cache file system 13 to continue reading the target resource through other means and increasing the probability of the cache file system 13 successfully reading the target resource.

[0152] If the cache file system 13 receives the above-mentioned third access response, optionally, for example, it can continue to read the target resource through other means.

[0153] For example, the cache file system 13 can obtain the target resource from the Object Storage Service (OSS) system in response to the third access response from the control device 14. Or, the cache file system 13 can also obtain the target resource from a network file system, etc. in response to the third access response from the control device 14. It should be understood that the present application does not limit the manner in which the cache file system 13 continues to obtain the target resource when receiving the third access response.

[0154] As a possible implementation, after obtaining the above-mentioned target resource, the cache file system 13 can, for example, also send a third access request to the control device 14. The third access request can be used to request writing the target resource into the above-mentioned cache manager 11.

[0155] Exemplarily, the third access request can include the above-mentioned target resource.

[0156] Correspondingly, the control device 14 can, for example, receive the third access request. Optionally, the control device 14 can, for example, in response to the third access request, add the identifier of the target data source corresponding to the control device 14 to the above-mentioned target resource.

[0157] Exemplarily, after adding the identifier of the target data source corresponding to the control device 14 to the target resource, the control device 14 can, for example, send a fourth access request to the cache manager 11. The fourth access request can include, for example, the target resource with the identifier of the target data source corresponding to the control device 14 added.

[0158] Correspondingly, the cache manager 11 can receive the fourth access request and store the target resource. By means of the identifier of the target data source corresponding to the control device 14, the cache manager 11 can isolate the target resource from other "data sources that are not the target data source corresponding to the control device 14" based on the identifier of the target data source corresponding to the control device 14.

[0159] In this embodiment, when the cache file system 13 receives the third access response, indicating that the target resource cannot be read from the cache manager 11, it can continue to obtain the above target resource through the object storage service system or the network file system, improving the probability of successfully reading the target resource. By sending a third access request to the control device 14, the cache file system 13 enables the control device 14 to write the target resource into the cache manager 11 based on the third access request, laying a foundation for the cache file system 13 to read the target resource from the cache manager 11 subsequently.

[0160] Figure 3 Another structural schematic diagram of the physical machine 10 provided by this application. As Figure 3 shown, the virtual machine instances 12 deployed on the physical machine 10 may include: a running container (Rund Pod) and a running virtual machine monitor (Rund VMM). Among them, Rund Pod includes: a cache file system 13 (such as cachefs fuse), a metadata engine (meta engine), and an OSS mount (i.e., OSS mount). Rund VMM includes: a control device 14 (such as a cachefs device), and a shared device 15 (virtio-fs device).

[0161] The physical machine 10 includes: a cache manager 11 (such as cachefs cache daemon), a memory cache / disk cache (such as mem / disk cache), and a sparse file. Among them, the sparse file (sparse file) can be used as a DAX window and is directly mapped into the virtual machine instance 12 through the virtio-fs channel. The control device 14 can be used to memory-map (Memory Mapping, mmap) the cache data to the corresponding DAX slot (slot).

[0162] As Figure 3As shown, the solid line between the cache file system 13, the metadata engine and the OSS mount represents the function call between the three. The dotted line connecting the cache file system 13, the control device 14, the cache manager 11 and the sparse file represents the remote procedure call (RPC, also known as RPC call) and the control link between the three. The double-dotted line between the cache file system 13, the shared device 15 and the sparse file represents the DAX data link between the three.

[0163] By adding a new control device 14 in rund vmm, the two aforementioned security multi-tenancy problems can be solved at the same time, and the control device 14 and the cachefs process in the virtual machine instance 12 can correspond one to one.

[0164] The cache file system 13 process in the virtual machine instance 12 listens to a port of vsock and waits for the control device 14 to connect, so as to avoid the process in the virtual machine instance 12 actively initiating requests to the outside. After the connection is established, the cache file system 13 can directly send the request to the control device 14. The control device 14 then performs security checks such as verification and current limiting on the request and then forwards it to the cache manager 11 on the physical machine 10.

[0165] In terms of data security isolation, when creating a control device 14, a data source name or tag (i.e., the identifier of the aforementioned target data source) can be set for each control device 14. After the first access request sent by the cache file system 13 is processed by the control device 14, the tag identifying the data source can be added to obtain a second access request, and then the second access request is forwarded to the cache manager 11. The cache manager 11 can isolate different data sources (such as different cache directories) by the identifier of the target data source to achieve secure multi-tenancy of data. For example, taking the data source that needs cache acceleration as an OSS mount, the OSS bucket can be used as the tag of the data source. Because the OSS bucket name is globally unique, the rund examples that can access the same bucket belong to the same user, or are authorized to other users for access, which improves the security of data access.

[0166] In addition, in order to reduce the performance impact caused by multiple data copies during request forwarding, each control device 14 can be configured with a corresponding new shared device 15. The shared device 15 can transparently transmit files to the virtual machine instance 12. Instead of the entire directory, this avoids the security risks caused by transparently transmitting a complete file system directory structure. By using the shared device 15 as a DAX window, the control device 14 can be used to map the required cached data into the DAX window and transparently transmit it to the process of the cached file system 13, achieving the purpose of directly accessing the cached content on the host in a zero-copy manner.

[0167] In this embodiment, by using the shared device 15 as a DAX window, the control device 14 can be used to map the required cached data into the DAX window without accessing the file content therein, realizing the on-demand mapping of target resources to the DAX window for use by the virtual machine instance 12. Utilizing the DAX data path of the shared device 15, the security risks of transparently transmitting the entire file system directory through the shared device 15 are avoided, and high-efficiency zero-copy data transmission between the virtual machine instance 12 and the physical machine 10 is achieved through DAX shared memory. By only transparently transmitting the target resources required by the cached file system 13 instead of the entire directory, the security risks of transparently transmitting the directory are reduced, and the security of data access is improved.

[0168] This application also provides a computer cluster. The computer cluster can include at least one physical machine 10 as described in any of the foregoing embodiments. The implementation principle and technical effects of the computer cluster provided in this application are similar to those of the foregoing physical machine 10, and will not be elaborated herein.

[0169] Figure 4 It is a schematic hardware structure diagram of an electronic device provided in this application. The electronic device can be the physical machine 10 as described in any of the foregoing embodiments. Figure 4 The illustrated electronic device 40 includes a memory 41, a processor 42, and a communication interface 43. The memory 41, the processor 42, and the communication interface 43 are communicatively connected to each other. For example, the memory 41, the processor 42, and the communication interface 43 can be communicatively connected by means of a network connection. Alternatively, the foregoing electronic device 40 can further include a bus 44. The memory 41, the processor 42, and the communication interface 43 are communicatively connected to each other through the bus 44. Figure 4 It is the electronic device 40 in which the memory 41, the processor 42, and the communication interface 43 are communicatively connected to each other through the bus 44.

[0170] The memory 41 can be a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM). The memory 41 can store a program. When the program stored in the memory 41 is executed by the processor 42, the processor 42 and the communication interface 43 are used to execute the data access method described in any of the foregoing embodiments. The memory can also store the data required by this data access method.

[0171] The processor 42 can be a general-purpose CPU, a microprocessor, an Application-Specific Integrated Circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits.

[0172] The processor 42 can also be an integrated circuit chip with the ability to process signals. In the implementation process, the data access method of this application can be completed by the integrated logic circuit in the hardware of the processor 42 or the instructions in software form. The above-mentioned processor 42 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments below of this application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments below of this application can be directly embodied as being executed and completed by the hardware decoding processor, or can be executed and completed by the combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 41, and the processor 42 reads the information in the memory 41 and combines its hardware to complete the data access method of this application.

[0173] The communication interface 43 uses a transceiver module such as, but not limited to, a transceiver to achieve communication between the electronic device 40 and other devices or communication networks. For example, a data set can be obtained through the communication interface 43.

[0174] When the above-mentioned electronic device 40 includes a bus 44, the bus 44 can include a path for transmitting information between the various components of the electronic device 40 (for example, the memory 41, the processor 42, the communication interface 43).

[0175] The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs. Specifically, program instructions are stored in the computer-readable storage medium, and the program instructions are used for the methods in the above embodiments.

[0176] The present application also provides a program product, which includes execution instructions stored in a readable storage medium. At least one processor of an electronic device can read the execution instructions from the readable storage medium, and the at least one processor executes the execution instructions to enable the electronic device to implement the data access methods provided by the above various embodiments.

[0177] The term "a plurality" herein refers to two or more. The term " / and" herein merely describes the associated relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after; in a formula, the character " / " represents a "division" relationship between the associated objects before and after. In addition, it should be understood that in the description of the present application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0178] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0179] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data access method, characterized in that, On the physical machine, there are deployed: a cache manager, and at least one virtual machine instance, where the virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device; the method includes: In response to a first access request for a target resource, the control device sends a second access request to the cache manager; the first access request is sent by the cache file system to the control device; the second access request carries: the name identifier of the target resource. The control device receives a first access response returned by the cache manager based on the second access request; the first access response carries: the index identifier corresponding to the target resource. The control device maps the index identifier to the corresponding shared device, and returns a second access response to the cache file system; the second access response includes: the virtual address space of the target resource in the shared device, so that the cache file system can access the target resource through the virtual address space.

2. The method according to claim 1, characterized in that, Different control devices correspond to different data sources, and the second access request also carries: the identifier of the target data source corresponding to the control device.

3. The method according to claim 2, characterized in that, The first access request carries: the name identifier of the target resource. In response to a first access request for a target resource, the control device sends a second access request to the cache manager, including: The control device performs a security check on the first access request. When the security check of the first access request passes, the control device adds the identifier of the target data source to the first access request to obtain the second access request. The control device sends the second access request to the cache manager.

4. The method according to any one of claims 1 - 3, characterized in that, The first access request is a read request. After the control device sends a second access request to the cache manager in response to a first access request for a target resource, the method further includes: In response to not obtaining the index identifier from the cache manager within a preset time period, the control device sends a third access response to the cache file system, and the third access response is used to indicate that the physical machine does not cache the target resource.

5. The method according to any one of claims 1 - 3, characterized in that, Before the control device sends a second access request to the cache manager in response to a first access request for a target resource, the method further includes: The control device establishes a connection with the cache file system through the corresponding virtual port.

6. A data access method, characterized in that, On the physical machine, there are deployed: a cache manager, and at least one virtual machine instance, where the virtual machine instance includes: a cache file system, a control device, and a shared device corresponding to the control device; the method includes: The shared device receives a data access request sent by the cache file system, and the data access request includes: the virtual address space of the target resource in the shared device; the virtual address space is obtained based on the method according to any one of claims 1-5. In response to the data access request, the shared device feeds back the target resource to the cache file system.

7. A data access method, characterized in that, A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes a cache file system, a control device, and a shared device corresponding to the control device. The method includes: The cache file system sends a first access request for a target resource to the control device, so that the control device, in response to the first access request, sends a second access request to the cache manager, and after receiving a first access response returned by the cache manager based on the second access request, maps an index identifier corresponding to the target resource to the corresponding shared device, and returns a second access response to the cache file system. The second access request carries: a name identifier of the target resource; the first access response carries: the index identifier; the second access response includes: a virtual address space of the target resource in the shared device; The cache file system receives the second access response and accesses the target resource through a direct access DAX channel between the cache file system and the shared device based on the virtual address space.

8. The method according to claim 7, characterized in that, Before the cache file system sends a first access request for a target resource to the control device, the method further includes: The cache file system listens to a virtual port corresponding to the control device to determine whether the control device establishes a connection through the virtual port. The cache file system sending a first access request for a target resource to the control device includes: The cache file system sends the first access request to the control device when a connection is established with the control device.

9. The method according to claim 7 or 8, characterized in that, If the first access request is a read request, after the cache file system sends a first access request for a target resource to the control device, the method further includes: The cache file system obtains the target resource from an object storage service system or a network file system in response to a third access response from the control device. The third access response is used to indicate that the physical machine does not cache the target resource. The third access response is sent by the control device to the cache file system in response to not obtaining the index identifier from the cache manager within a preset time period.

10. The method according to claim 9, characterized in that, After the cache file system obtains the target resource from an object storage service system or a network file system in response to a third access response from the control device, the method further includes: The cache file system sends a third access request to the control device. The third access request is used to request writing the target resource into the cache manager.

11. A data access method, characterized in that, A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes a cache file system, a control device, and a shared device corresponding to the control device. The method includes: The cache manager receives a second access request sent by the control device, where the second access request is sent by the control device in response to a first access request for a target resource, and the first access request is sent by the cache file system to the control device. The second access request carries: the name identifier of the target resource. The cache manager obtains the index identifier corresponding to the target resource based on the name identifier of the target resource. The cache manager sends a first access response carrying the index identifier to the control device, so that the control device maps the index identifier to the corresponding shared device and returns a second access response to the cache file system. The second access response includes: the virtual address space of the target resource in the shared device, so that the cache file system accesses the target resource through the virtual address space.

12. The method according to claim 11, wherein, The first access request is a read request, and the method further includes: When the cache manager fails to query the index identifier corresponding to the target resource based on the name identifier of the target resource, it sends a third access response to the cache file system through the control device, and the third access response is used to indicate that the physical machine does not cache the target resource.

13. A data access method, wherein, A cache manager and at least one virtual machine instance are deployed on a physical machine. The virtual machine instance includes: a cache file system, a control device, and the shared device corresponding to the control device. The method includes: The cache file system sends a first access request for a target resource to the control device. The control device sends a second access request to the cache manager in response to the first access request. The second access request carries: the name identifier of the target resource. The cache manager obtains the index identifier corresponding to the target resource based on the name identifier of the target resource. The cache manager sends a first access response to the control device, and the first access response carries: the index identifier. The control device maps the index identifier to the corresponding shared device and returns a second access response to the cache file system. The second access response includes: the virtual address space of the target resource in the shared device. In response to the second access response, the cache file system accesses the target resource based on the virtual address space through the direct access DAX channel between the cache file system and the shared device.

14. A physical machine, wherein, A cache manager and at least one virtual machine instance are deployed on the physical machine. The virtual machine instance includes: a cache file system, a control device, and the shared device corresponding to the control device. Among them, the control device is used to execute the method according to any one of claims 1-5. The shared device is used to execute the method according to claim 6. The cache file system is used to execute the method according to any one of claims 7-10. The cache manager is used to execute the method according to claim 11 or 12.

15. A computer cluster, wherein, The computer cluster includes at least one physical machine according to claim 14.

16. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer-executable instructions, which are used to implement the method described in any one of claims 1-5, 6, 7-10, 11-12, and 13 when executed by a processor.

17. A computer program product, wherein, The computer program product includes a computer program, which implements the method described in any one of claims 1-5, 6, 7-10, 11-12, and 13 when executed by a processor.

Images