A file storage method and device, electronic equipment and storage medium

Abstract

The embodiment of the present application provides a file storage method and device, comprising: creating a virtual machine image file under a cloud platform, dividing the virtual machine image file into a plurality of file segments according to a file size of the virtual machine image file and a set strip size, respectively performing division processing on each file segment, so that each file segment after the division processing comprises at least one specific structure object; and storing the specific structure object into a plurality of nodes of a strip-based distributed file system, so that after the larger virtual machine image file is divided into the specific structure object under the cloud platform, a distributed strip storage mode of the strip-based distributed file system can be used, thereby improving the high availability of the virtual machine image file.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a file storage method, a file storage device, an electronic device and a readable storage medium. BACKGROUND

[0002] A KVM (Kernel-based Virtual Machine) image is managed by an operating system in the form of a file, and thus the KVM image is also referred to as an image file. The image file includes a raw (a file extension) format and a qcow2 (a file extension) format. The raw format is simple and has good performance, but the raw format does not support a sparse format and needs the support of a file system in the operating system to support the sparse format. The qcow2 format can make the image file smaller when the file system in the operating system does not support the sparse format, and the qcow2 format supports Copy-on-write and supports maintaining multiple snapshots.

[0003] The storage mode of a virtual machine image file generated by a CPU based on a part of a specific instruction system platform includes local storage, NFS (Network File System) storage, commercial storage, etc. The local storage and the NFS storage have problems such as insufficient storage space, single point of failure, and large access disk IO bottleneck defects, and the commercial storage has the problem of high price.

[0004] Based on the above reasons, for occasions where a large image file needs to be quickly expanded, has good stability, and is stored for a long time, especially a virtual machine image file under a cloud computing platform, the virtual machine image file is usually large, and the high availability requirement of the large image file cannot be met. SUMMARY

[0005] The technical problem to be solved by the embodiments of the present application is to provide a file storage method, device, electronic device and readable storage medium, so as to solve the problem that the high availability requirement of a large image file cannot be met.

[0006] In order to solve the above problems, the present application provides a file storage method, comprising:

[0007] Under a cloud platform, a virtual machine image file is created;

[0008] According to the file size of the virtual machine image file and the set strip size, the virtual machine image file is divided into a plurality of file segments;

[0009] respectively, each of the file segments includes at least one specific structure object; wherein the specific structure object includes a pointer member and an integer member, the pointer member is used to point to a position where data to be processed in the file segment is stored, and the integer member is used to record a byte number of the data to be processed in the file segment;

[0010] The specific structure object is stored in a plurality of nodes of the strip-based distributed file system.

[0011] Optionally, the dividing the virtual machine image file into a plurality of file segments according to the file size of the virtual machine image file and a set strip size includes:

[0012] The virtual machine image file is parsed to obtain the file size;

[0013] According to the strip size and the file size, the number of file segments into which the virtual machine image file is divided and the data amount of each file segment are calculated.

[0014] Optionally, the respectively dividing each of the file segments includes:

[0015] According to the number of file segments and the data amount of each file segment, a file segment to be uploaded this time is determined;

[0016] The file segment to be uploaded this time is divided into at least one specific structure object, and a corresponding memory space is allocated to each specific structure object;

[0017] The specific structure object is stored in a plurality of nodes of the strip-based distributed file system.

[0018] A target node is scheduled to obtain the specific structure object of the file segment to be uploaded this time from the memory space allocated to each specific structure object, and the specific structure object is stored;

[0019] After the file segment to be uploaded this time is stored, a processing step of a next file segment to be uploaded is started until all the file segments to be uploaded are stored.

[0020] Optionally, the determining the file segment to be uploaded this time according to the number of file segments and the data amount of each file segment includes:

[0021] According to the number of file segments and the data amount of each file segment, a starting position of the file segment to be uploaded this time in the virtual machine image file is determined;

[0022] According to the start position and the strip size, a strip volume index and a storage block index are calculated;

[0023] The scheduling target node obtains the specific structure object of the file segment to be uploaded this time from the memory space allocated by each specific structure object, and stores the specific structure object, and the method comprises the steps of:

[0024] According to the strip volume index and the storage block index, the specific structure object of the file segment to be uploaded this time is written into the corresponding storage position.

[0025] Correspondingly, the application also provides a file storage device, which comprises:

[0026] A file creation module is configured to create a virtual machine image file under a cloud platform.

[0027] A file segment division module is configured to divide the virtual machine image file into a plurality of file segments according to a file size of the virtual machine image file and a set strip size.

[0028] An object division module is configured to perform division processing on each file segment respectively, so that each file segment after the division processing comprises at least one specific structure object; wherein the specific structure object comprises a pointer member and an integer member, the pointer member is used for pointing to a position where data to be processed in the file segment is stored, and the integer member is used for recording a byte number of the data to be processed in the file segment.

[0029] An object storage module is configured to store the specific structure object into a plurality of nodes of a strip-based distributed file system.

[0030] Optionally, the file segment division module comprises:

[0031] An analysis sub-module is configured to analyze the virtual machine image file to obtain the file size.

[0032] A calculation sub-module is configured to calculate a number of file segments of the virtual machine image file and a data amount of each file segment according to the strip size and the file size.

[0033] Optionally, the object division module comprises:

[0034] A file segment determination sub-module is configured to determine a file segment to be uploaded this time according to the number of file segments and the data amount of each file segment.

[0035] An object division sub-module is configured to divide the file segment to be uploaded this time into at least one specific structure object, and allocate a corresponding memory space for each specific structure object.

[0036] The object storage module comprises;

[0037] The storage submodule is used for scheduling a target node, obtaining the specific structure object of the file segment to be uploaded this time from the memory space allocated by each specific structure object, and storing the specific structure object.

[0038] The cycle submodule is used for starting the processing step of the next file segment to be uploaded after the storage of the file segment to be uploaded this time is completed, until all the file segments to be uploaded are stored.

[0039] Optionally, the file segment determination submodule comprises:

[0040] The position determination unit is used for determining the starting position of the file segment to be uploaded this time in the virtual machine image file according to the number of file segments and the data amount of each file segment.

[0041] The index calculation unit is used for calculating the strip volume index and the storage block index according to the starting position and the strip size.

[0042] The storage submodule comprises:

[0043] The storage unit is used for writing the specific structure object of the file segment to be uploaded this time into the corresponding storage position according to the strip volume index and the storage block index.

[0044] Correspondingly, the present application further provides an electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete the communication among each other through the communication bus.

[0045] The memory is used for storing a computer program.

[0046] The processor is used for executing the program stored on the memory, and realizes the method steps of any of the above.

[0047] Correspondingly, the present application further provides a readable storage medium, when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device can execute the above file storage method.

[0048] According to the embodiment of the present application, by creating a virtual machine image file under a cloud platform, the virtual machine image file is divided into a plurality of file segments according to the file size of the virtual machine image file and a set strip size, each of the file segments is processed separately, and each of the processed file segments includes at least one specific structure object; the specific structure object includes a pointer member and an integer member, the pointer member is used to point to a position where data to be processed in the file segment is stored, and the integer member is used to record the byte number of the data to be processed in the file segment; and the specific structure object is stored in a plurality of nodes of a strip-based distributed file system. After the larger virtual machine image file is divided into specific structure objects under the cloud platform, the distributed strip storage mode of the strip-based distributed file system can be used, thereby improving the high availability of the virtual machine image file. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A step flow chart of a file storage method according to an embodiment of the present application is shown;

[0050] Figure 2 A step flow chart of a file storage method according to an embodiment of the present application is shown;

[0051] Figure 3 A flow diagram of distributed strip writing storage of a virtual machine image file is shown;

[0052] Figure 4 A structure block diagram of a file storage device according to an embodiment of the present application is shown;

[0053] Figure 5 A structure block diagram of an electronic device for file storage according to an exemplary embodiment is shown. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned objects, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0055] Reference Figure 1 A step flow chart of a file storage method according to an embodiment of the present application is shown, which can specifically include the following steps:

[0056] Step 101, creating a virtual machine image file under a cloud platform.

[0057] In the embodiment of the present application, the cloud platform, also referred to as a cloud computing platform, refers to a hardware-based service that has the ability to provide computing, network and storage. The virtual machine (Virtual Machine) refers to a complete computer system that is simulated by software and runs in a completely isolated environment. The cloud platform is composed of virtual machines, and data is stored in virtual disks, that is, virtual machine image files. The most obvious feature of the virtual machine image file is a large file, and the virtual machine image file represents the virtual hard disk of the virtual machine, and the capacity is generally more than 10G. For the strategy of multiple redundant backups of small files or general files, the space resources occupied by multiple redundant backups of large files are too much, so the storage strategy for small files or general files is not suitable for virtual machine image files.

[0058] In the embodiment of the present application, under the platform based on instruction system A, a virtual machine image file in qcow2 format is created. The virtual machine image file of the instruction system A platform is completed by the interface qemu-img create and qemu-kvm of KVM and QEMU (Quick EMUlator, virtual operating system emulator) in the Linux kernel under the instruction system A. The qemu-img create interface is a virtual disk file creation interface provided by QEMU. The qemu-kvm interface is an interface that provides virtualization of processors, memories and input / output devices, and manages the creation and calling of various virtual devices. The above instruction system A is any existing instruction system, such as the LoongArch instruction system.

[0059] The main components of the image file in qcow2 format are as follows:

[0060] Header (header): QCowHeader structure, stores the version information of Qcow (a file extension), the offset position and size information of the L1 table in the qcow2 file, cluster information, image file size, refcount table and snapshot offset position, etc.

[0061] L1 table: each table entry describes the related information of an L2 table entry in the L2 table, and the size is 64 bits.

[0062] L2 table: each table entry describes the attributes of a cluster, and the size is 64 bits.

[0063] Refcount table: used to save the first-level allocation table of cluster, and the size is 64bit.

[0064] One or more refcount blocks: used to save the second-level allocation table of cluster, and each refcount block occupies one cluster.

[0065] Snapshot: used to save the snapshot header, and each table item describes the information related to snapshot, and the length is variable.

[0066] Cluster: the entire mirror file is managed in the unit of cluster, and the metadata such as qcow2 format header and refcount table are stored in cluster.

[0067] In step 102, the virtual machine image file is divided into multiple file segments according to the file size of the virtual machine image file and the set strip size.

[0068] In the embodiment of the application, the file size of the virtual machine image file can be obtained from the virtual machine image file. For example, the format header of the virtual machine image file in qcow2 format includes the file size.

[0069] In the embodiment of the application, strip is a method of dividing continuous data into data blocks of the same size and writing each segment of data to different disks in an array. In simple terms, strip is a method of combining multiple disk drives into a volume. In many cases, the above process is completed by a hardware controller. The volume, also known as a logical volume, is a disk management method, which is a virtual disk formed by a logical disk, and can also be referred to as a disk partition. The strip volume is a volume composed of the free space of multiple disk drives. After striping, the speed provided by the strip volume is much faster than that provided by a single disk, and striping is used to realize I / O load sharing of the system. The strip size, also known as the strip depth, refers to the size of the strip data block written on each disk. The data block size of RAID (Redundant Arrays of Independent Disks) is generally between 2KB and 512KB (or larger), and the value is a power of 2, that is, 2KB, 4KB, 8KB, 16KB, etc.

[0070] In the embodiment of the present application, when setting the strip size, there is no universally applicable optimal strip size. If the strip size is reduced, the file is divided into more smaller data blocks. These data blocks will be stored on more hard disks, thus improving the transmission performance, but the performance of disk positioning is reduced due to multiple searches for different data blocks. If the strip size is increased, contrary to reducing the strip size, the transmission performance is reduced and the positioning performance is improved. According to different application types, different performance requirements, and different characteristics of different drives, the strip size that meets the requirements can be set, and any applicable strip size can be set, and the embodiment of the present application does not limit this.

[0071] In the embodiment of the present application, according to the file size and the strip size, the virtual machine image file can be divided into multiple file segments. The size of each file segment should be less than or equal to the strip size, and the sum of the sizes of all file segments is equal to the file size. Any applicable division method can be used, for example, the virtual machine image file is parsed to obtain the file size, the number of file segments and the data amount of each file segment of the virtual machine image file are calculated according to the strip size and the file size, or other any applicable division method, and the embodiment of the present application does not limit this. For example, the strip size is 128 KB, and the file size is 257 KB. The file can be divided into three file segments, the first file segment and the second file segment are both 128 KB, and the third file segment is 1 KB.

[0072] In step 103, each file segment is processed respectively to make each file segment after the processing include at least one specific structure object; wherein the specific structure object includes a pointer member and an integer member, the pointer member is used to point to the position of the data to be processed in the file segment, and the integer member is used to record the byte number of the data to be processed in the file segment.

[0073] In the embodiment of the present application, the storage is organized in a specific structure. The specific structure object includes a pointer member and an integer member, the pointer member is used to point to the position of the data to be processed in the file segment, and the integer member is used to record the byte number of the data to be processed in the file segment. The specific structure object can be an iovec (a structure related to read and write operations) structure object, which defines one or more vector elements. Generally, the specific structure object can be an array of multiple elements; for each transmitted element, the pointer member in the specific structure object points to a buffer storing the data received by readv (read operation) or the data to be sent by writev (write operation); and the integer member in the specific structure object respectively determines the maximum length of the received data and the length of the actually written data in various cases.

[0074] In the embodiment of the present application, before storing the data, the data needs to be organized into a structure convenient for uploading. For each file segment, the file segment is divided into at least one specific structure object. The maximum value of the integer member of the specific structure object is set according to the parameters of the hardware such as the processor and the memory. According to the size of the file segment and the integer member, the number of the specific structure objects divided from the file segment can be determined.

[0075] In step 104, the specific structure object is stored in the plurality of nodes of the strip-based distributed file system.

[0076] In the embodiment of the present application, when the capacity of a single file (i.e. the occupied storage space) is very large and the number of clients is more, the strip volume cannot meet the demand, at this time, the file management is combined with distribution and striping, that is, a strip-based distributed file system is used, and the performance of the file system is related to the number of servers. The strip-based distributed file system is composed of a plurality of nodes, and each node is a server. The strip-based distributed file system can effectively solve the problem of data storage and management, and expand a file system fixed in a certain place to any number of places / multiple file systems, and a plurality of nodes form a file system network. Each node can be distributed in different places and communicate and transmit data between nodes through a network. When using the strip-based distributed file system, it is not necessary to care about where the data is stored or where the data is obtained, and only the data in the file system needs to be managed and stored like using a local file system.

[0077] The implementation mode of the distributed storage of the specific structure object into the plurality of nodes of the strip-based distributed file system can include various modes; optionally, the corresponding node is allocated according to the LRU (Least Recently Used) strategy, and then the specific structure object is uploaded and distributed into the plurality of nodes one by one.

[0078] In a specific application scenario, a virtual machine image file is created on a KVM virtual machine cloud platform. All storage servers or other storage devices with storage space are put into a storage pool. If a storage pool containing three servers is to be created, the other two servers are added to the storage pool from the first server server1. The virtual machine image file is mounted to a volume of a strip-based distributed file system (such as GlusterFS); the volume is organized in a distributed strip mode to set a path to organize the volume; by checking the storage space in the GlusterFS volume and the existence of the instance, it can be seen that the space of any server in the GlusterFS volume under the set path is the sum of the spaces of the three servers, and the virtual machine exists on any server, but the actual physical storage exists on one or a few servers. In addition, when the storage space is flexibly increased, the GlusterFS Gluster volume add-brick (Gluster volume add-brick) and Gluster volume rebalance (Gluster volume rebalance) interfaces can be used to realize.

[0079] According to the embodiment of the application, by creating a virtual machine image file under a cloud platform, the virtual machine image file is divided into a plurality of file segments according to the file size of the virtual machine image file and the set strip size, each file segment is processed separately, and each processed file segment includes at least one specific structure object; wherein the specific structure object includes a pointer member and an integer member, the pointer member is used to point to the position of the data to be processed in the file segment, and the integer member is used to record the byte number of the data to be processed in the file segment. The specific structure object is stored in a plurality of nodes of a strip-based distributed file system, so that after the larger virtual machine image file is divided into specific structure objects under the cloud platform, the distributed strip storage mode of the strip-based distributed file system can be used, thereby improving the high availability of the virtual machine image file.

[0080] Referring to Figure 2 , a step flowchart of a file storage method according to the second embodiment of the application is shown, which can specifically include the following steps:

[0081] Step 201, a virtual machine image file is created under a cloud platform.

[0082] In the embodiments of the present application, the specific implementation of this step can refer to the description in the foregoing embodiments, which will not be described here again.

[0083] In step 202, the virtual machine image file is parsed to obtain the file size.

[0084] In the embodiments of the present application, the information of the file size is included in the virtual machine image file, and the file size is obtained by parsing the virtual machine image file. For example, the file size is included in the format header of the virtual machine image file in the qcow2 format.

[0085] In step 203, the number of file segments and the data amount of each file segment of the virtual machine image file are calculated according to the stripe size and the file size.

[0086] In the embodiments of the present application, the step of dividing the virtual machine image file into multiple file segments is not actually divided into multiple files, but the number of file segments and the data amount of each file segment are calculated.

[0087] For example, Figure 3 is a flowchart of the distributed stripe write storage of the virtual machine image file. First, the GlusterFS performs context initialization to obtain global variables, for example, page_size (page size) = 128*1024ull, that is, 128KB. Then, the context parameters of the xlator are obtained, including FOP (fuse_operation, operation parameter), atmoic (atomicity parameter), gluster_ctx (gluster context parameter), etc. Among them, the xlator is a highly modularized component with a well-defined internal structure, including structure body and interface function prototype definition. The parameters are assigned initial values, such as stripe size, file size, etc. The volume of the stripe type is created, and the file segmentation and stripe grouping are performed according to the stripe size and the file size. By default, a virtual machine image file is divided into multiple file segments according to 128KB, and the file size of each file segment is 128KB from the first file segment. However, the file size of the last file segment is the total file size of the uploaded virtual machine image file (n*128KB+xKB), and the content actually stored in the first part (n*128KB) of the last file segment is the binary data for padding, and the content actually stored in the latter part (xKB) is the valid data.

[0088] In step 204, the file segment to be uploaded this time is determined according to the number of file segments and the data amount of each file segment.

[0089] In the embodiment of the present application, one file segment is uploaded at a time when uploading a file. According to the number of file segments and the data amount of each file segment, the file segment to be uploaded this time can be determined. Specifically, various implementation manners can be included, for example: according to the number of file segments and the data amount of each file segment, the starting position of the file segment to be uploaded this time in the virtual machine image file is determined; according to the starting position and the size of a strip, the strip volume index and the storage block index are calculated. For example, as shown in Figure 3 After entering the loop part, polling storage is performed. For the file segment to be uploaded this time, the index is calculated and obtained, including the strip volume index and the index of the storage block.

[0090] Step 205, the file segment to be uploaded this time is divided into at least one specific structure object, and each specific structure object is allocated with a corresponding memory space.

[0091] In the embodiment of the present application, each specific structure object is allocated with a corresponding memory space, the starting position of which is determined by the pointer member of the specific structure object, and the size of which is determined by the integer member of the specific structure object. The memory space is used to store the specific structure object. The memory space is also applied according to the size of the specific structure object. One specific structure object needs one memory space corresponding to the structure object. For each file segment, the file segment is divided into how many specific structure objects, how many such memory spaces need to be allocated.

[0092] Step 206, the target node is scheduled to obtain the specific structure object of the file segment to be uploaded this time from the memory space allocated by each specific structure object, and the specific structure object is stored.

[0093] In the embodiment of the present application, the target node in the strip-based distributed file system is scheduled to store the specific structure object of this round of loop. Specifically, the target node obtains the specific structure object of the file segment to be uploaded this time from the memory space allocated by each specific structure object, and then stores the specific structure object. For example, as shown in Figure 3 The sub-node in the GlusterFS system is scheduled to store data.

[0094] In an optional embodiment of the present application, according to the number of file segments and the data amount of each file segment, a specific implementation of determining the file segment to be uploaded this time can include: according to the number of file segments and the data amount of each file segment, determining the starting position of the file segment to be uploaded this time in the virtual machine image file; according to the starting position and the stripe size, calculating the stripe volume index and the storage block index, and corresponding, the target node is scheduled, the specific structure object of the file segment to be uploaded this time is obtained from the memory space allocated by each specific structure object, and the specific structure object is stored. In a specific implementation, it can include: according to the stripe volume index and the storage block index, the specific structure object of the file segment to be uploaded this time is written into the corresponding storage position.

[0095] In the above embodiment, the stripe volume adopts a polling mode, and according to the stripe size, when the specific structure object fills the stripe on the current volume, the stripe on the next volume of the current volume is polled until the stripes of all volumes are filled, and then the stripe of the first volume is returned. Since the data amount in the specific structure object in the file segment to be uploaded this time and the stripe size are known, the corresponding stripe volume index of the file segment to be uploaded this time can be calculated.

[0096] In the above embodiment, the storage block index idx = (((offset+offset_offset) / local->stripe_size) % fctx->stripe_count). In the above embodiment, offset+offset_offset is the starting position of the data to be stored this time in the virtual machine image file, local->stripe_size is the stripe size of the node, and fctx->stripe_count represents the number of stripes called.

[0097] According to the stripe volume index and the storage block index, the storage position of the file segment stored each time can be determined. Each specific structure object is stored one by one, and the storage position of the specific structure object stored each time can be determined. After the specific structure object is uploaded to the target node, it is stored in the corresponding storage position.

[0098] Step 207, after the file segment to be uploaded this time is stored, the processing step of the next file segment to be uploaded is started, and the processing of all file segments to be uploaded is completed.

[0099] In the above embodiment, after the file segment to be uploaded this time is stored, it is checked whether all file segments have been uploaded, and if there are file segments to be uploaded, the processing of the next file segment to be uploaded is started. That is, the next loop is entered, and the process returns to step 204, and the processing of all file segments to be uploaded is completed.

[0100] For example, as shown in FIG. 3, after the scheduling sub-node completes the storage of the current file segment, it is checked whether there is data to be uploaded, and if there is a file segment to be uploaded, the index of the next file segment is calculated, and the next polling storage is started. Figure 3

[0101] According to the embodiment of the present application, by creating a virtual machine image file under a cloud platform, parsing the virtual machine image file to obtain the file size, calculating the number of file segments and the data amount of each file segment of the virtual machine image file according to the stripe size and the file size, determining the file segment to be uploaded this time according to the number of file segments and the data amount of each file segment, dividing the file segment to be uploaded this time into at least one specific structure object, and respectively allocating a corresponding memory space for each specific structure object, the target node is scheduled to obtain the specific structure object of the file segment to be uploaded this time from the memory space allocated by each specific structure object, and the specific structure object is stored. After the storage of the file segment to be uploaded this time is completed, the processing step of the next file segment to be uploaded is started until all the file segments to be uploaded are stored, so that after the larger virtual machine image file is divided into specific structure objects under the cloud platform, the distributed stripe storage mode based on the distributed file system of the stripe can be used, thereby improving the high availability of the virtual machine image file.

[0102] It should be noted that, for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the present application is not limited by the described action sequence, because according to the embodiment of the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiment of the present application.

[0103] Referring to FIG. 4, a structural block diagram of a file storage device embodiment of the embodiment three of the present application is shown, which can specifically include the following modules: Figure 4

[0104] The file creation module 301 is used to create a virtual machine image file under a cloud platform.

[0105] The file segment division module 302 is used to divide the virtual machine image file into a plurality of file segments according to the file size of the virtual machine image file and the set stripe size.

[0106] ​​The object division module 303 is configured to perform division processing on each of the file segments respectively, so that each of the file segments after the division processing comprises at least one specific structure object; the specific structure object comprises a pointer member and an integer member; the pointer member is used to point to a position where data to be processed in the file segment is stored; and the integer member is used to record a byte number of the data to be processed in the file segment.

[0107] The object storage module 304 is configured to store the specific structure object into a plurality of nodes of the strip-based distributed file system.

[0108] Optionally, the file segment division module comprises:

[0109] The parsing sub-module is configured to parse the virtual machine image file to obtain the file size.

[0110] The calculation sub-module is configured to calculate the number of file segments into which the virtual machine image file is divided and the data amount of each file segment according to the strip size and the file size.

[0111] Optionally, the object division module comprises:

[0112] The file segment determination sub-module is configured to determine a file segment to be uploaded this time according to the number of file segments and the data amount of each file segment.

[0113] The object division sub-module is configured to divide the file segment to be uploaded this time into at least one specific structure object, and allocate a corresponding memory space to each specific structure object.

[0114] The object storage module comprises:

[0115] The storage sub-module is configured to schedule a target node, acquire the specific structure object of the file segment to be uploaded this time from the memory space allocated to each specific structure object, and store the specific structure object.

[0116] The loop sub-module is configured to start a processing step of a next file segment to be uploaded after the storage of the file segment to be uploaded this time is completed, until all the file segments to be uploaded are stored.

[0117] Optionally, the file segment determination sub-module comprises:

[0118] The position determination unit is configured to determine a starting position of the file segment to be uploaded this time in the virtual machine image file according to the number of file segments and the data amount of each file segment.

[0119] The index calculation unit is configured to calculate a strip volume index and a storage block index according to the starting position and the strip size.

[0120] The storage sub-module comprises:

[0121] The storage unit is configured to write the specific structure object of the file segment to be uploaded this time into a corresponding storage location according to the tape volume index and the storage block index.

[0122] According to the embodiment of the present application, by creating a virtual machine image file under a cloud platform, dividing the virtual machine image file into a plurality of file segments according to the file size of the virtual machine image file and a set strip size, and respectively performing division processing on each file segment, each file segment after the division processing includes at least one specific structure object. The specific structure object includes a pointer member and an integer member. The pointer member is used to point to a position where data to be processed in the file segment is stored, and the integer member is used to record the byte number of the data to be processed in the file segment. The specific structure object is stored in a plurality of nodes of a strip-based distributed file system. After the larger virtual machine image file is divided into specific structure objects under the cloud platform, the distributed strip storage mode of the strip-based distributed file system can be used, thereby improving the high availability of the virtual machine image file.

[0123] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts can be referred to the part of the method embodiment.

[0124] Figure 5 is a structural block diagram of an electronic device 700 for file storage according to an exemplary embodiment. For example, the electronic device 700 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0125] Referring to Figure 5 , the electronic device 700 can include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0126] The processing component 702 generally controls the overall operations of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 can include one or more processors 720 to execute instructions to complete the steps of the methods described above, in whole or in part. Moreover, the processing component 702 can include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 can include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0127] The memory 704 is configured to store various types of data to support the operations of the electronic device 700. Examples of these data include instructions to operate any applications or methods on the electronic device 700, contact data, phonebook data, messages, pictures, videos, and the like. The memory 704 can be realized by any type of volatile or non-volatile memory devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disc, or optical disc.

[0128] The power component 704 provides power to the various components of the electronic device 700. The power component 704 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 700.

[0129] The multimedia component 708 includes a screen to provide an output interface between the electronic device 700 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensors can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 708 includes a front camera and / or a back camera. The front camera and / or the back camera can receive external multimedia data when the electronic device 700 is in an operating mode, such as a shooting mode or a video mode. Each of the front camera and the back camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0130] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC) that is configured to receive an external audio signal when the electronic device 700 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0131] The I / O interface 712 provides an interface between the processing component 702 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0132] The sensor component 714 includes one or more sensors for providing status assessments of various aspects of the electronic device 700. For example, the sensor component 714 can detect an open / closed position of the device 700, relative positioning of components, such as a display and a keypad of the electronic device 700, a change of position of the electronic device 700 or a component of the electronic device 700, presence or absence of user contact with the electronic device 700, orientation or acceleration / deceleration of the electronic device 700, and a temperature change of the electronic device 700. The sensor component 714 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 714 can also include a light sensor, such as a CMOS or CCD image sensor, for use in an imaging application. In some embodiments, the sensor component 714 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0133] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an example embodiment, the communication component 714 receives a broadcast signal or broadcast-related information from an external broadcasting management system via a broadcasting channel. In an example embodiment, the communication component 714 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technology.

[0134] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-described methods.

[0135] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, for example, the memory 704 including instructions, is also provided, which can be executed by the processor 720 of the electronic device 700 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0136] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a terminal, enables the terminal to perform a file storage method, the method comprising:

[0137] Under a cloud platform, a virtual machine image file is created;

[0138] According to a file size of the virtual machine image file and a set strip size, the virtual machine image file is divided into a plurality of file segments;

[0139] Each of the file segments is respectively processed, so that each of the processed file segments includes at least one specific structure object; wherein the specific structure object includes a pointer member and an integer member, the pointer member is used to point to a position where data to be processed in the file segment is stored, and the integer member is used to record a byte number of the data to be processed in the file segment;

[0140] The specific structure object is stored in a plurality of nodes of a strip-based distributed file system.

[0141] Optionally, the dividing the virtual machine image file into a plurality of file segments according to the file size of the virtual machine image file and the set strip size comprises:

[0142] The virtual machine image file is parsed to obtain the file size;

[0143] According to the strip size and the file size, a number of file segments into which the virtual machine image file is divided and a data amount of each file segment are calculated.

[0144] Optionally, the processing each of the file segments respectively, so that each of the processed file segments includes at least one specific structure object, comprises:

[0145] determining the file segment to be uploaded this time according to the number of file segments and the data amount of each file segment;

[0146] dividing the file segment to be uploaded this time into at least one specific structure object, and respectively allocating a corresponding memory space to each specific structure object;

[0147] storing the specific structure object into a plurality of nodes of the strip-based distributed file system, including:

[0148] scheduling a target node to obtain the specific structure object of the file segment to be uploaded this time from the memory space allocated to each specific structure object, and storing the specific structure object;

[0149] after the file segment to be uploaded this time is stored, starting the processing step of the next file segment to be uploaded until all the file segments to be uploaded are stored.

[0150] Optionally, the determining the file segment to be uploaded this time according to the number of file segments and the data amount of each file segment includes:

[0151] determining the starting position of the file segment to be uploaded this time in the virtual machine image file according to the number of file segments and the data amount of each file segment;

[0152] calculating a strip volume index and a storage block index according to the starting position and the strip size;

[0153] the scheduling a target node to obtain the specific structure object of the file segment to be uploaded this time from the memory space allocated to each specific structure object, and storing the specific structure object includes:

[0154] writing the specific structure object of the file segment to be uploaded this time into a corresponding storage position according to the strip volume index and the storage block index.

[0155] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other.

[0156] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer program instructions.

[0157] Embodiments of the present application are described herein with reference to the Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing one or more functions specified in a flow or multiple flows and / or blocks.

[0158] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing one or more functions specified in a flow or multiple flows and / or blocks.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing one or more functions specified in a flow or multiple flows and / or blocks.

[0160] While preferred embodiments of the present application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the foregoing description. Therefore, the appended claims are intended to encompass within their scope all such variations and modifications as are included within the scope of the present application.

[0161] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or terminal device. Without more limitations, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0162] The above describes in detail the file storage method and file storage device provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A file storage method characterized by comprising: The method comprises the following steps: creating a virtual machine image file under a cloud platform; dividing the virtual machine image file into a plurality of file segments according to a file size of the virtual machine image file and a set strip size; determining a starting position of a file segment to be uploaded this time in the virtual machine image file according to a quantity of the plurality of file segments obtained after the virtual machine image file is divided and data amounts of the respective file segments; calculating a strip volume index and a storage block index according to the starting position and the strip size; dividing the file segment to be uploaded this time into at least one specific structure object, and respectively allocating a corresponding memory space to each specific structure object; wherein the specific structure object comprises a pointer member and an integer member, the pointer member is used to point to a position where data to be processed in the file segment is stored, and the integer member is used to record a byte number of the data to be processed in the file segment; storing the specific structure object into a plurality of nodes of a strip-based distributed file system according to the strip volume index and the storage block index.

2. The method of claim 1, wherein, The step of dividing the virtual machine image file into a plurality of file segments according to a file size of the virtual machine image file and a set strip size comprises the following steps: parsing the virtual machine image file to obtain the file size; calculating a quantity of the file segments of the virtual machine image file divided and data amounts of the respective file segments according to the strip size and the file size.

3. The method of claim 2, wherein, The step of storing the specific structure object into a plurality of nodes of a strip-based distributed file system comprises the following steps: scheduling a target node to obtain the specific structure object of the file segment to be uploaded this time from the memory space allocated to each specific structure object, and storing the specific structure object; after the file segment to be uploaded this time is stored, starting a processing step of a next file segment to be uploaded until all the file segments to be uploaded are stored.

4. The method of claim 3, wherein, The step of scheduling a target node to obtain the specific structure object of the file segment to be uploaded this time from the memory space allocated to each specific structure object, and storing the specific structure object comprises the following steps: writing the specific structure object of the file segment to be uploaded this time into a corresponding storage position according to the strip volume index and the storage block index.

5. A file storage device, characterized by The method comprises the following steps: a file creating module is configured to create a virtual machine image file under a cloud platform; a file segment dividing module is configured to divide the virtual machine image file into a plurality of file segments according to a file size of the virtual machine image file and a set strip size; An object division module is configured to determine a starting position of a file segment to be uploaded this time in the virtual machine image file according to a number of file segments obtained after division of the virtual machine image file and data amounts of the file segments; calculate a stripe volume index and a storage block index according to the starting position and a stripe size; divide the file segment to be uploaded this time into at least one specific structure object, and respectively allocate a corresponding memory space to each specific structure object; wherein the specific structure object comprises a pointer member and an integer member, the pointer member is configured to point to a position where data to be processed in the file segment is stored, and the integer member is configured to record a byte number of the data to be processed in the file segment; An object storage module is configured to store the specific structure object into a plurality of nodes of a stripe-based distributed file system according to the stripe volume index and the storage block index.

6. The apparatus of claim 5, wherein, The file segment division module comprises: An analysis submodule is configured to analyze the virtual machine image file to obtain the file size; A calculation submodule is configured to calculate a number of file segments obtained after division of the virtual machine image file and data amounts of the file segments according to the stripe size and the file size.

7. The apparatus of claim 6, wherein, The object storage module comprises: A storage submodule is configured to schedule a target node, acquire the specific structure object of the file segment to be uploaded this time from the memory space allocated to each specific structure object, and store the specific structure object; A loop submodule is configured to start a processing step of a next file segment to be uploaded after storage of the file segment to be uploaded this time is completed, until storage of all the file segments to be uploaded is completed.

8. The apparatus of claim 7, wherein, The storage submodule comprises: A storage unit is configured to write the specific structure object of the file segment to be uploaded this time into a corresponding storage position according to the stripe volume index and the storage block index.

9. An electronic device, comprising: The electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the communication bus; The memory is configured to store a computer program; The processor is configured to execute the program stored on the memory to implement the method steps in any one of claims 1-4.

10. A readable storage medium, characterized by, When instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the file storage method in one or more of claims 1-4.

Images