Distributed object storage method, device and equipment, and computer storage medium
By using storage area collection identification and erasure coding technology in distributed object storage systems, the limitations of SMR disks in random writes and in-place updates are solved, the disk space utilization is improved, and efficient storage of SMR disks is achieved manage.
Patent Information
- Application Number
- CN202110907042.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-09
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-08-09
AI Technical Summary
Existing distributed object storage systems are difficult to effectively support tiled magnetic recording (SMR) disks, especially in terms of random writes and in-situ update writes, and cannot efficiently utilize disk space.
By introducing storage area collection identification in the distributed object storage system, the management node allocates storage areas of multiple storage node devices to the target object, and improves disk space utilization through erasure coding technology, and realizes efficient storage management of SMR disks.
It realizes effective storage management of SMR disks, improves disk space utilization, supports erasure coding technology, and solves the problem that existing systems cannot efficiently utilize disk space.
Smart Images

Figure CN113672175B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to the field of distributed storage technology, and provides a distributed object storage method, apparatus and device, and a computer storage medium. Background Art
[0002] Shingled Magnetic Recording (SMR) disk is a leading next-generation disk technology. The adjacent tracks of SMR disks partially overlap in sequence, resulting in its read behavior being the same as that of ordinary hard disk drives (HDD). However, due to the characteristic of partial overlap between tracks in sequence, the data of overlapping tracks will be overwritten when writing. Therefore, SMR disks do not support random writes and in-place update writes, and only support sequential writes from the beginning to the end.
[0003] However, it is precisely because of the physical characteristics of SMR disks, that is, adjacent tracks partially overlap in sequence, that more tracks can be accommodated in the same area, so that SMR disks have a higher storage density per unit storage medium, and correspondingly, the storage cost is lower. Therefore, SMR disks have an unavoidable advantage in price. It is imperative to support SMR disks in distributed object storage systems. However, there is currently no better solution to the problem of how to support SMR disks in distributed object storage systems. Summary of the invention
[0004] The embodiments of the present application provide a distributed object storage method, apparatus, device and computer storage medium for implementing storage management of SMR disks in a distributed object storage system.
[0005] In one aspect, a distributed object storage method is provided, which is applied to a terminal device included in a distributed object storage system, and the method includes:
[0006] Based on the target object to be stored, sending a storage space application request to the management node device, wherein the storage space application request carries object attribute information of the target object;
[0007] Receive the storage area set identifier returned by the management node device in response to the storage space application request, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to the storage area set identifier belong; the storage area set identifier is generated by the management node device according to the object attribute information, and is used to uniquely identify the storage area set storing the target object;
[0008] Based on the network identifiers of the N storage node devices, shard data write requests are sent to the N storage node devices respectively, each shard data write request carries the storage area set identifier and is used to request to store a shard data of the target object in the corresponding storage node device.
[0009] In one aspect, a distributed object storage method is provided, which is applied to a management node device included in a distributed object storage system, and the method includes:
[0010] Generate a storage area set identifier of the target object to be stored based on the object attribute information carried in the storage space application request sent by the terminal device, wherein the storage area set identifier uniquely identifies a storage area set storing the target object;
[0011] Based on a preset load balancing strategy, N storage node devices are selected from a plurality of storage node devices included in the distributed object storage system, wherein the N target storage areas corresponding to the storage area set identifiers are derived from the N storage node devices, and one storage area is derived from one storage node device;
[0012] Returning the storage area set identifier and the network identifiers of the N storage node devices to the terminal device;
[0013] Receive the storage area attribute information reported by each of the N storage node devices, and update the storage mapping relationship stored in itself based on the storage area attribute information, the storage area attribute information is used to indicate the storage area identifier associated with the storage area set identifier, and the storage mapping relationship is used to indicate the storage area attribute information corresponding to each storage area set identifier.
[0014] In one aspect, a distributed object storage method is provided, which is applied to a storage node device included in a distributed object storage system, and the method includes:
[0015] Receive a shard data write request sent by a terminal device, wherein the shard data write request carries a storage area set identifier of a target object to be stored, and the shard data write request carries a shard data of the target object;
[0016] Based on the shard data write request and the usage of the storage disk included in the storage disk, a target storage area is selected from the storage disk included in the storage disk, and the shard data is stored in the target storage area; and
[0017] Associating the storage area set identifier with the target storage area;
[0018] Reporting storage area attribute information to a management node device, wherein the storage area attribute information is used to indicate a storage area identifier of the target storage area associated with the storage area set identifier in the storage node device.
[0019] In one aspect, a distributed object storage device is provided, which is applied to a terminal device included in a distributed object storage system, and the device includes:
[0020] A request unit, configured to send a storage space application request to a management node device based on a target object to be stored, wherein the storage space application request carries object attribute information of the target object;
[0021] A receiving unit, configured to receive a storage area set identifier returned by the management node device in response to the storage space application request, and network identifiers of N storage node devices to which N target storage areas corresponding to the storage area set identifier belong; the storage area set identifier is generated by the management node device according to the object attribute information, and is used to uniquely identify a storage area set storing the target object;
[0022] A data writing unit is used to send shard data writing requests to the N storage node devices respectively based on the network identifiers of the N storage node devices, each shard data writing request carries the storage area set identifier and is used to request to store a shard data of the target object in the corresponding storage node device.
[0023] Optionally, the request unit is further used to:
[0024] Based on the target file to be stored, sending a file creation request to the management node device, wherein the file creation request is used to request to assign a file identifier to the target file;
[0025] Receiving a file identifier returned by the management node device in response to the file creation request, wherein the file identifier uniquely identifies the target file;
[0026] Based on the target object included in the target file, the storage space application request is sent to the management node device, and the object attribute information carried by the storage space application request includes the file identifier and the object identifier of the target object.
[0027] Optionally, the request unit is further used to:
[0028] Based on the target data type to be stored, sending a storage space creation request to the management node device, wherein the storage space creation request is used to request allocation of a storage space identifier for itself;
[0029] Receiving a storage space identifier returned by the management node device in response to the storage space creation request, wherein the storage space identifier uniquely identifies data of a target data type output by the terminal device;
[0030] Optionally, the device further includes a file reading unit;
[0031] The request unit is further used to: send a file retrieval request to the management node device based on the name of the target file to be read;
[0032] The receiving unit is further configured to receive the storage area set identifiers of each object returned by the management node device in response to the file retrieval request, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to each storage area set identifier belong;
[0033] The file reading unit is used to send a shard data acquisition request to N storage node devices corresponding to each of the objects, wherein the shard data acquisition request carries a storage area set identifier of the object; and
[0034] The shard data returned by each storage node device in response to the corresponding shard data acquisition request is received, and the corresponding object data is obtained based on the shard data of each object to obtain a target file composed of the object data of each object.
[0035] Optionally, the storage space application request further carries a data encoding type used to store the target object, and the number N of storage node devices is determined according to the data encoding type;
[0036] The file writing unit is specifically used for:
[0037] Based on the data encoding type, encode the target object to obtain N slice data of the target object;
[0038] Based on the network identifiers of the N storage node devices, the obtained N shard data are respectively carried in shard data write requests and sent to corresponding storage node devices.
[0039] In one aspect, a distributed object storage device is provided, which is applied to a management node device included in a distributed object storage system, and the device includes:
[0040] An identifier generating unit, configured to generate a storage area set identifier of a target object to be stored based on object attribute information carried in a storage space application request sent by a terminal device, wherein the storage area set identifier uniquely identifies a storage area set storing the target object;
[0041] A storage node management unit, configured to select N storage node devices from a plurality of storage node devices included in the distributed object storage system based on a preset load balancing strategy, wherein the N target storage areas corresponding to the storage area set identifiers originate from the N storage node devices, and one storage area originates from one storage node device;
[0042] a transceiver unit, configured to return the storage area set identifier and the network identifiers of the N storage node devices to the terminal device; and
[0043] Receive the storage area attribute information reported by each of the N storage node devices, and update the storage mapping relationship stored in itself based on the storage area attribute information, the storage area attribute information is used to indicate the storage area identifier associated with the storage area set identifier, and the storage mapping relationship is used to indicate the storage area attribute information corresponding to each storage area set identifier.
[0044] Optionally, the identification generating unit is further used to:
[0045] In response to the file creation request sent by the terminal device, assigning a file identifier to the target file, wherein the file identifier uniquely identifies the target file; the target file includes the target object;
[0046] The storage area set identifier is generated based on the file identifier and the object identifier of the target object.
[0047] Optionally, the device further comprises a retrieval unit, configured to:
[0048] Based on the target file name carried in the file retrieval request sent by the terminal device, querying the storage area set identifiers of each object included in the target file in the storage mapping relationship, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to each storage area set identifier belong;
[0049] The storage area set identifiers of the respective objects and the network identifiers of the N storage node devices corresponding to each storage area set identifier are sent to the terminal device.
[0050] Optionally, the storage mapping relationship is cached in the memory of the management node device, and the storage mapping relationship is stored in a plurality of sub-tables, each sub-table storing a portion of the storage mapping relationship;
[0051] Then the retrieval unit is specifically used for:
[0052] The plurality of subtables are traversed and queried respectively in a parallel processing manner to obtain storage area set identifiers of the respective objects.
[0053] In one aspect, a distributed object storage device is provided, which is applied to a storage node device included in a distributed object storage system, and the device includes:
[0054] A receiving unit, configured to receive a shard data write request sent by a terminal device, wherein the shard data write request carries a storage area set identifier of a target object to be stored, and the shard data write request carries one shard data of the target object;
[0055] A storage area management unit, configured to select a target storage area from the storage disks included in the unit based on the shard data write request and the usage of the storage disks included in the unit, and store the shard data in the target storage area; and associate the storage area set identifier with the target storage area;
[0056] The sending unit is used to report storage area attribute information to the management node device, wherein the storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the storage area set identifier in the storage node device.
[0057] Optionally, the storage disk is a shingled magnetic recording SMR disk.
[0058] On the one hand, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.
[0059] In one aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.
[0060] In one aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above methods.
[0061] In the embodiment of the present application, when storing the target object, it is necessary to apply for storage space from the management node device, the management node device allocates a storage node device for it, and generates a storage area set identifier of the target object based on the object attribute information of the target object to be stored, so that the terminal device initiates shard data storage to the storage node device based on the storage area set identifier, the storage node device allocates the storage area on the storage disk included in itself for the target object, and binds the storage area with the storage area set identifier, and reports to the management node device at the same time, and the management node device updates the mapping relationship of the storage area set identifier. Therefore, in the embodiment of the present application, the entire object storage process is realized by taking the storage area set as the management unit, and using the storage area set identifier as a transfer flow to realize the storage area management between the terminal device, the management node device and the storage node device. At the same time, when storing the object, multiple storage areas provided by multiple storage node devices are allocated to it at the same time, and each storage area stores a shard data, so as to realize the support of the erasure code technology by the distributed object storage system, so that the disk space utilization rate of the storage system is higher. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0063] Figure 1 An application scenario diagram provided for an embodiment of the present application;
[0064] Figure 2 A schematic diagram of the system architecture of a distributed object storage system provided in an embodiment of the present application;
[0065] Figure 3 A schematic diagram of a process flow of a distributed object storage method provided in an embodiment of the present application;
[0066] Figure 4 A schematic diagram of the composition of ZG provided in an embodiment of the present application;
[0067] Figure 5 Another schematic diagram of the distributed object storage method provided in the embodiment of the present application;
[0068] Figure 6 A flowchart of a file reading process provided by an embodiment of the present application;
[0069] Figure 7 A schematic diagram of the structure of a distributed object storage device provided in an embodiment of the present application;
[0070] Figure 8 Another structural diagram of a distributed object storage device provided in an embodiment of the present application;
[0071] Fig. 9 A schematic diagram of another structure of a distributed object storage device provided in an embodiment of the present application;
[0072] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0073] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0074] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:
[0075] SMR disk: SMR disk divides the tracks into multiple bands, that is, the continuous writing area composed of continuous tracks. Each area is a basic unit that needs to be written sequentially. Band is the physical concept of SMR disk, and the corresponding logical concept is called zone. The size of a zone is generally 256MB. The reading and writing of data in the zone only supports sequential writing, and does not support random writing, that is, it moves backward in sequence according to the position of the write pointer, but each zone is independent and does not affect each other. Since SMR disks still ensure that data can be read from the non-overlapping part of the track, data reading within the zone can still be performed randomly. SMR disks are divided into partitions during manufacturing, and the starting position of the zone is marked by leaving a large gap between the zones.
[0076] Storage area set (zone group): a set consisting of storage areas of disks on different storage node devices, which is the storage unit of the distributed storage system of the embodiment of the present application. Each storage area set is identified by a unique storage area set identifier (zone group ID, ZGID).
[0077] Data redundancy protection strategy: In a distributed object storage system, in order to improve data reliability and ensure data security, it is generally necessary to adopt a data redundancy protection strategy for data redundancy. There are two common data redundancy protection strategies: multi-copy mode and erasure coding (EC) mode. In the multi-copy mode, data blocks are stored in different places to achieve redundant backup. Taking 3 copies as an example, the disk space utilization rate is only 33%. The redundancy is relatively high, which will undoubtedly increase the consumption of storage space and increase storage costs, but it has the characteristics of being safer and faster to read. In the EC mode, real data is encoded by the erasure coding algorithm to obtain redundancy, and the real data and redundant data are stored in the system together to achieve the purpose of data redundancy. Because erasure coding needs to calculate the check value for data reading and writing, there is a high central processing unit (CPU) resource overhead, but erasure coding can improve the space utilization of the storage system and reduce storage costs. Taking the common EC mode 4+1 as an example, the system's disk space utilization rate can reach 80%, which is much higher than the utilization rate of the existing commonly used 3 copies, and has its own cost advantage that cannot be ignored.
[0078] When using erasure coding technology for storage, the original data is first sharded, then backup data is generated based on the sharding code, and finally the original data and backup data are written to different storage media. Data recovery is also called the decoding process, which is the inverse operation of encoding. In order to be able to recover data, the encoding method must be reversible.
[0079] Distributed storage: It is a storage method that stores data in multiple independent devices. Traditional network storage systems use centralized storage servers to store all data. The storage server becomes the bottleneck of system performance and the focus of reliability and security. It cannot meet the needs of large-scale storage applications. Distributed network storage systems use a scalable system structure and use multiple storage servers to share the storage load, which not only improves the system's reliability, availability and access efficiency, but also makes it easy to expand.
[0080] Object: An object is the smallest storage unit in a distributed object storage system. Each object has a unique object ID.
[0081] File: A file can be composed of multiple objects, each of which is identified by a unique file ID. For example, for a video file, each object can be a segment in the video file.
[0082] Bucket: It can be understood as a logical folder, a container for storing objects. For example, for a distributed object storage system for security surveillance videos, a video bucket can be created for a surveillance device to store videos shot by the surveillance device. The videos shot by the surveillance device over a period of time can be formed into video files, and the video files are divided into multiple video objects for storage in the distributed object storage system. In addition, an image bucket can be created for the surveillance device to store images shot by the surveillance device. In other words, each bucket can uniquely correspond to a surveillance device and is used to store a type of data of the surveillance device, and each bucket is identified by a bucket ID.
[0083] Metadata: also known as intermediary data or relay data, it is data about data. It is mainly information describing data properties, used to support functions such as indicating storage location, historical data, resource search, and file records. Metadata is a kind of electronic directory. In order to achieve the purpose of compiling a directory, it is necessary to describe and collect the content or characteristics of the data, so as to achieve the purpose of assisting data retrieval. In a distributed storage system, first of all, metadata can provide user-based information. For example, metadata that records the business description information of a data item can help users use the data. Secondly, metadata can support the system's management and maintenance of data. For example, metadata about the storage method of a data item can support the system to access data in the most efficient way.
[0084] The following is a brief introduction to the design concept of the embodiments of the present application.
[0085] There is currently no mature distributed storage system that supports SMR disks, but SMR disks have unparalleled cost advantages in actual project deployment, so the implementation of SMR disks is imperative.
[0086] In addition, the current mainstream distributed storage systems, such as Hadoop Distributed File System (HDFS), etc., use the multi-copy mode to improve data security. Although there are systems that support the EC mode, the EC shard data stored in them are all based on standard HDD disks or SSD disks based on standard file systems for data reading and writing, and there is no solution for how to manage SMR disk zones. There is no difference between the multi-copy mode and the EC mode itself, but they are suitable for different business scenarios. For example, in the Internet field, the unit data value density of stored data is high, so the multi-copy mode is more inclined to be used. For the security field, because the unit data value density of surveillance videos in the security field is not as high as that of the Internet, the extremely low storage space utilization rate of the multi-copy mode is unacceptable to users.
[0087] Based on this, the embodiment of the present application provides a distributed object storage method, apparatus and device and computer storage medium. In this method, when storing a target object, it is necessary to apply for storage space from a management node device, the management node device allocates a storage node device to it, and generates a storage area set identifier of the target object based on the object attribute information of the target object to be stored, so that the terminal device initiates shard data storage to the storage node device based on the storage area set identifier, the storage node device allocates a storage area on the storage disk included in itself to the target object, and binds the storage area to the storage area set identifier, and reports to the management node device at the same time, and the management node device updates the mapping relationship of the storage area set identifier.
[0088] Therefore, in the embodiment of the present application, the entire object storage process is realized by taking the storage area set as the management unit and identifying the storage area set as a transmission flow for realizing storage area management among the terminal device, the management node device and the storage node device, thereby realizing the storage and management of objects in the Zone of the SMR disk in the distributed object storage system.
[0089] At the same time, when performing object storage, multiple storage areas provided by multiple storage node devices are allocated to it at the same time, and each storage area stores a shard of data, thereby realizing the support of the erasure code technology by the distributed object storage system, making the disk space utilization rate of the storage system higher.
[0090] In addition, in the embodiment of the present application, the distributed object storage system is designed based on ZoneGroup, and a group of newly initialized Zones are associated through ZGID, and the shard data corresponding to the erasure code is written to this group of Zones in batches and concurrently, and the ability to carry out unified load and management based on the ZG level is provided, and the ability to support erasure code technology is realized through the policy management of the native SMR disk Zone space. At the same time, the management node ZG global cache data is not persistent, and the management node constructs and maintains the ZG cache purpose through the real-time and periodic Zone reporting of the storage node, that is, the management node caches the used ZG of the entire storage system in memory state to realize the real-time management and update function of all SMR disk Zones in the system. In order to solve the problem of the large number of ZGs in large-scale systems, the internal ZG cache of the management node performs memory state sub-table hash storage to provide more efficient query and traversal performance, and the management node realizes the read and write function of real-time business data by controlling the generation of ZGID and the transparent transmission of ZGID inside each module.
[0091] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.
[0092] The solution provided in the embodiment of the present application can be applied to most distributed object storage scenarios, especially to distributed object storage systems in the security field. Figure 1 As shown, it is an application scenario diagram provided by an embodiment of the present application. In this scenario, it includes a terminal device 101, a management node device 102 and a storage node device 103.
[0093] The terminal device 101 is any device that needs to store data, such as a mobile phone, a tablet computer (PAD), a laptop computer, a desktop computer, a smart TV, and a monitoring device. A software development kit (SDK) for storage and retrieval can be installed on the terminal device 101, wherein the SDK encapsulates the external interfaces of the distributed object storage system to call these interfaces to achieve interaction with the distributed object storage system. The SDK can be a separate client or embedded in any client.
[0094] The management node device 102 is used to provide metadata management services for the distributed object storage system, respond to metadata requests for the entire object storage system, and store the metadata image of the entire system internally. It is also responsible for the load balancing of the storage node device 103, including some other file recovery control logic. The management node device 102 can be, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers. For example, the management node device 102 can help expand the main device and the backup device to improve the reliability of the system management service.
[0095] The storage node device 103 provides real file storage services, provides external data stream write response, manages disks, and periodically scans and reports data blocks. Each storage node device 103 may include multiple physical storage disks, such as SMR disks, and each physical storage disk includes multiple storage areas.
[0096] In one possible implementation, the distributed object storage system may be a storage system deployed for a monitoring system, and the terminal device 101 may be a monitoring device deployed in each monitoring area, and the SDK has been set up on these monitoring devices. When the monitoring device shoots monitoring videos, the SDK can be called to store these monitoring videos on each storage node device 103.
[0097] In another possible implementation, the distributed object storage system may be a storage system deployed for a monitoring system, and the terminal device 101 may be connected to the monitoring device to receive the video stream captured by the monitoring device, and call its own SDK to store the video stream on each storage node device 103.
[0098] The terminal device 101, the management node device 102 and the storage node device 103 may be directly or indirectly connected to each other through one or more networks 104. The network 104 may be a wired network or a wireless network, for example, a mobile cellular network or a wireless fidelity (Wireless-Fidelity, WIFI) network, or other possible networks, which are not limited in the embodiments of the present application.
[0099] It should be noted that in the embodiment of the present application, the number of terminal devices 101 can be one or more, and similarly, the number of storage node devices 103 can be one or more, that is, there is no limitation on the number of terminal devices 101 or storage node devices 103.
[0100] Of course, the method provided in the embodiment of the present application is not limited to Figure 1The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here.
[0101] See also Figure 2 , which is a schematic diagram of the system architecture of a distributed object storage system provided in an embodiment of the present application, wherein the distributed object storage system includes an SDK client, a management node (Meta Data Service, MDS) and multiple storage nodes (data node, DN).
[0102] The SDK client can be deployed in a terminal device, that is, a terminal device that has a need to access the object storage system, such as a monitoring device or a terminal device connected to a monitoring device.
[0103] The management node is deployed in the management node device, and is used to manage the metadata of the entire object storage system, respond to the metadata requests of the entire object storage system, and store the metadata image of the entire system internally. It is also responsible for the load balancing of the storage nodes, including some other file recovery control logic. Generally speaking, in order to improve the reliability of data management, the management node can include a master node and one or more backup nodes.
[0104] The storage node provides file storage services, external data stream write response, disk management, and periodic data block scanning and reporting.
[0105] When performing object storage, the access interface provided by the SDK client is called to apply for storage space from the management node, and the object attribute information of the object to be stored, such as the object identifier and file identifier, is carried. The management node allocates a storage node for it based on the load balancing strategy, and generates a corresponding storage area set identifier ZGID based on the object attribute information. ZGID is used to uniquely identify the storage area set of the object. The management node returns the ZGID to the SDK client, and at the same time returns the network identifier of the allocated storage node, such as the Internet Protocol (IP) address.
[0106] The SDK client initiates real data storage to each storage node based on the network identifier and carries the ZGID. When each storage node receives the ZGID, it selects a storage area of a physical storage disk based on its own load balancing strategy, associates it with the ZGID, and writes the object to be stored to the disk. At the same time, it also reports the association between the storage area and the ZGID to the management node. The management node updates the mapping relationship between its own storage and the ZGID so that the object can be retrieved later based on the mapping relationship.
[0107] See also Figure 2 As shown in the figure, the interaction between the SDK client and the management node, and between the management node and the storage node is the signaling flow. The signaling flow is the information flow that does not involve object data, and is mainly the information transmission related to the system function in the object storage system. The interaction between the SDK client and the storage node is the data flow. The data flow is the information flow for storing and reading real object data, that is, writing data to the storage node, or reading data from the storage node. For the convenience of distinguishing, Figure 2 The information flow and data flow are shown in different ways.
[0108] Next, combine Figure 1 Application scenarios and Figure 2 The distributed object storage method provided in the embodiment of the present application is introduced in the system architecture of Figure 3 , is a flow chart of a distributed object storage method provided in an embodiment of the present application, which can be performed by Figure 1 The method is performed by combining various devices, and the process of the method is described as follows.
[0109] Step 301: The SDK client sends a storage space application request to the MDS, and the MDS receives the storage space application request.
[0110] In the embodiment of the present application, when object storage is required, the SDK can be called to implement the object storage process.
[0111] Specifically, the SDK client may send a storage space application request to the management node device based on the target object to be stored. The storage space application request carries object attribute information of the target object. The object attribute information may include, for example, an object ID and a file ID.
[0112] For example, taking the surveillance video storage for security monitoring as an example, after the surveillance device shoots the surveillance video, it needs to store the surveillance video segment by segment in the object storage system. For example, a 10-minute surveillance video file A can be further divided into 5 video objects B for storage. Then for each video object B, when storing it, you can use the SDK client to initiate a storage space application request to MDS, and carry the file ID of surveillance video file A and the object ID of video object B in the request. The file ID is the file ID that the SDK client requests MDS to allocate for surveillance video file A before storing surveillance video file A, and it can uniquely identify surveillance video file A. Similarly, an object ID uniquely identifies a video object B of surveillance video file A.
[0113] Step 302: The MDS generates a storage area set identifier for the target object and allocates a DN list to it.
[0114] In an embodiment of the present application, after receiving a storage space application request, the MDS generates a storage area set identifier ZGID for the target object based on the object attribute information carried therein, and the ZGID uniquely identifies the storage area set (Zone Group, ZG) storing the target object.
[0115] For example, when the object attribute information includes file ID and object ID, MDS encapsulates the file ID and object ID of the space application to obtain a ZG tag type, namely, ZGID. ZGID uniquely identifies a ZoneGroup.
[0116] See also Figure 4 As shown, a schematic diagram of the composition of a ZG provided for the implementation of this application is shown, wherein each storage node is provided with multiple disk slots, each of which is used to set a physical storage disk Disk, for example, Disk1 to Diskn can be placed on DN1, and each physical storage disk includes multiple storage areas. Taking the 4+1 EC mode as an example, that is, an object stores 5 shard data, then the ZG of an object requires 5 storage areas Zone, that is Figure 4 Zone 1 to Zone 5 shown in the figure, each zone comes from a physical storage disk on a storage node, such as Figure 4 The ZG shown includes Zone1 to Zone5 which are derived from DN1 to DN5 respectively.
[0117] In the embodiment of the present application, in order to improve the space utilization of the disk, an SMR disk with higher space utilization is used, so each storage area can be a zone of the SMR disk. Of course, a disk with similar characteristics to the SMR disk can also be used.
[0118] In the embodiment of the present application, the storage space application request also carries the data encoding type used to store the target object, and the number N of storage node devices is determined according to the data encoding type. The data encoding type specifically refers to the EC mode used. In specific applications, the number N of zones inside the ZG can be dynamically adjusted when applying for space according to different EC modes. That is to say, based on different EC modes, the MDS will allocate ZoneGroups containing different numbers for it. For example, the EC mode used by SDK client A is the 8+1 EC mode, and the EC mode used by SDK client B is the 4+1 EC mode. Then the ZoneGroup allocated to the storage object of SDK client A contains 9 zones, and the ZoneGroup allocated to the storage object of SDK client B contains 5 zones.
[0119] In the embodiment of the present application, the MDS will also allocate a DN list for the target object. The DN list contains the network identifiers of the corresponding number of storage nodes. For example, Figure 4 For example, the DN list includes the network identifiers of 5 storage nodes.
[0120] Specifically, the MDS may select N storage nodes from multiple DNs included in the distributed object storage system based on a preset load balancing strategy, wherein the N target storage areas corresponding to the ZGID originate from the N storage nodes, and one storage area originates from one storage node.
[0121] Using the above example, the MDS can select five DNs with lower loads from all DNs based on a certain load balancing strategy, such as a polling strategy, a weight selection strategy, or a load selection strategy, so that the zones included in these five DNs form a ZoneGroup.
[0122] However, it should be noted that although the MDS has allocated the ZGID and DN list, at the current stage it can only clarify the DN list to which the zone corresponding to the ZGID belongs, but it has not yet clarified which specific zones the ZGID corresponds to.
[0123] Step 303: The MDS returns the ZGID and DN list to the SDK client.
[0124] Step 304: The SDK client sends a shard data write request to each of the N storage node devices.
[0125] In an embodiment of the present application, when the SDK client obtains the ZGID and DN list, it can obtain the network identifiers of the N nodes storing the target object, and then based on the network identifiers, send shard data write requests to the N storage node devices respectively. Each shard data write request carries the ZGID and is used to request that a shard data of the target object be stored in the corresponding storage node device.
[0126] Specifically, the SDK client encodes the target object based on the adopted data encoding type, that is, it slices the target object to obtain N shard data of the target object, and then based on the network identifiers of the N DNs, carries the obtained N shard data in the shard data write request and sends them to the corresponding DN.
[0127] Continuing with the above example, the DN list can carry the IP addresses of five storage nodes. Then the SDK client can split the target object into five shards and initiate data stream writes to these IP addresses in parallel to write the five shards of the target object into these DNs respectively.
[0128] Step 305: The DN selects a target storage area based on the usage of the storage disks included in the DN, and associates the storage area set identifier with the target storage area.
[0129] In an embodiment of the present application, during the data stream writing process, each DN responds to the above-mentioned shard data write request, selects a storage area on the storage disk according to its own disk load, associates the ZGID with the target storage area, and stores this association locally. Then, when a subsequent search is performed, the corresponding storage area can be found based on the ZGID.
[0130] Specifically, when DN receives a write request for a ZGID, it first locally queries whether there is a mapping relationship for this ZGID. If not, it selects a new Zone to associate with it based on certain load election rules to achieve the association between Zone and ZGID.
[0131] Step 306: DN writes the slice data of the target object into the storage area.
[0132] For example, when the storage disk is an SMR disk, the DN may sequentially write the data stream in the Zone of the SMR disk based on the sequential writing characteristic of the SMR disk.
[0133] Step 307: DN reports storage area attribute information to the MDS. The storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the ZGID in the storage node device.
[0134] Based on the above description, after MDS allocates DN, it only knows the DN associated with ZGID, but cannot clearly identify the specific Zone on DN. Therefore, after DN allocates Zone for the target object, it is necessary to report the Zone to MDS to update the storage area attribute information written by this data stream in MDS, that is, to update the storage mapping relationship stored in MDS. The storage mapping relationship is used to indicate the storage area attribute information corresponding to each storage area set identifier. At this point, the storage process of a target object is completed.
[0135] Specifically, MDS, as the metadata manager of the entire object storage system, manages the metadata of the entire object storage system, including the above-mentioned storage mapping relationship. Here we mainly take the storage mapping relationship as an example to introduce it. The storage mapping relationship is centered on ZGID and associates and stores all information related to the ZGID, such as the file ID, objectID, DN list, Zone identifier corresponding to the ZGID, and the size and type of the target object.
[0136] It should be noted that Figure 3Only DN1 executing 305 to 307 is used as an example to illustrate, but other DNs will also execute these processes. Since the execution processes are similar, they will not be described in detail.
[0137] In the embodiment of the present application, in order to improve the efficient query of the storage mapping relationship and save the storage space of the MDS, the storage mapping relationship stored on the MDS is not persistently stored or serialized in the management node, but the storage mapping relationship is stored in the memory to achieve the cache of the storage mapping relationship in the memory state. Then, the construction of the storage mapping relationship on the MDS needs to rely on the Zone report of the DN, so that the MDS maintains and updates the storage mapping relationship cached in the MDS in real time according to the results of the Zone report.
[0138] Specifically, the zone report of DN may include the following situations:
[0139] (1) Periodic reporting (non-real-time reporting)
[0140] In an embodiment of the present application, each DN also maintains the storage mapping relationship related to its own node, so the reporting period of each DN can be set in advance, and each DN can report its own storage mapping relationship to the MDS according to the set period, so that the MDS can maintain and update the storage mapping relationship.
[0141] Specifically, the periodic reporting of DN nodes can include incremental reporting and full reporting. Incremental reporting means that DN periodically reports the newly added storage mapping relationship, and full reporting means that DN reports the storage mapping relationship stored by itself. In actual applications, the periods of incremental reporting and full reporting can be reasonably set according to specific circumstances.
[0142] (2) Real-time business reporting (real-time reporting)
[0143] In the embodiment of the present application, real-time service reporting means that after each data stream is written, the DN reports the storage area attribute information of the data stream written to the MDS, that is, the process of the above step 306.
[0144] Therefore, MDS realizes the load and management of all zones in the entire system through periodic incremental and full reporting, as well as real-time business reporting. For example, after the object storage system restarts MDS, it gradually builds the storage mapping relationship of the internal memory state storage of MDS based on the incremental and full reporting of DN; or, when DN restarts registration, DN will make the first zone report to MDS, and MDS will update the internal memory state cache based on the reported information. Through the interaction of the above two signaling flows, the correctness of the ZG information cache inside MDS is guaranteed, and the correctness and accuracy of MDS load balancing is guaranteed.
[0145] In the embodiments of the present application, it is also considered that the possible cluster scale of the entire distributed object storage system is usually large. For example, in terms of the usual system scale, the storage nodes can reach as many as four or five hundred nodes, each DN has 36 disk slots, and each SMR disk contains 70,000 zones, that is, when the storage capacity of each SMR disk is 14TB, the number of corresponding storage mapping relationships is obviously very large. If such a large storage mapping relationship is stored based on a single table, then during query traversal, due to the huge amount of data in a single table, it is very easy to cause high latency in operations such as query, affecting the user experience. Therefore, in order to further reduce the response latency, when storing the storage mapping relationship, multiple sub-tables are used for storage, and each sub-table stores part of the storage mapping relationship.
[0146] Specifically, when storing, each storage mapping relationship is hashed according to zgId, and stored in each sub-table partition (HashMap) in the memory state. In this way, when querying and managing subsequently, each sub-table can be queried and managed independently and in parallel, thereby improving response efficiency.
[0147] In the embodiment of the present application, before step 301, steps S1 to S4 may also be included. Figure 5 The figure shows a flow chart of steps S1 to S4.
[0148] S1: The SDK client sends a storage space creation request to the MDS.
[0149] Specifically, the SDK client sends a storage space creation request to the MDS based on the target data type to be stored. The storage space creation request is used to request allocation of a storage space identifier (bucket ID) for itself.
[0150] S2: MDS returns the bucket ID to the SDK client.
[0151] Taking the object storage system corresponding to security monitoring as an example, when storing surveillance videos for each surveillance device, in order to distinguish the videos of different surveillance devices, it is necessary to assign a different identifier to each surveillance device. Therefore, before the surveillance device stores the surveillance video, its corresponding SDK client needs to initiate a storage space (bucket) creation request to MDS to request that a bucket ID for storing surveillance videos be allocated to the surveillance device. This is equivalent to the SDK client requesting to create a logical folder for the surveillance device to store the surveillance video of the surveillance device.
[0152] A bucket ID uniquely identifies the storage space corresponding to a data type of a monitoring device. For example, a monitoring device can request to store monitoring videos and monitoring pictures, so MDS can allocate two bucket IDs, corresponding to monitoring videos and monitoring pictures respectively.
[0153] S3: The SDK client sends a file creation request to the MDS.
[0154] Specifically, the SDK client sends a file creation request to the MDS based on the target file to be stored, so as to request allocation of a file ID for the target file.
[0155] S4: MDS returns the file ID to the SDK client.
[0156] Based on the SDK client's request, MDS generates a corresponding file ID for the target file based on the target file's information and returns it to the SDK client. The file ID uniquely identifies the target file. In this way, when storing the objects included in the target file later, the file ID can be carried in the request.
[0157] In the embodiment of the present application, the distributed object storage system also provides a file reading method, see Figure 6 The figure is a flowchart of the file reading process provided in an embodiment of the present application.
[0158] Step 601: The SDK client sends a file retrieval request to the MDS.
[0159] The file retrieval request carries the name of the target file to be read.
[0160] Generally speaking, when a user searches for a file through an upper layer application, the user usually enters the name of the file as the search field. For example, for a surveillance video, the user can enter the time of the surveillance video as the file name for search.
[0161] Step 602: Based on the target file name, the MDS searches the storage mapping relationship for the ZGID corresponding to each object included in the target file, and the DN list corresponding to each ZGID.
[0162] In the embodiment of the present application, since the MDS adopts a sub-table storage method, when querying the target file, a parallel processing method can be used to traverse and query multiple sub-tables cached by the MDS respectively to obtain the ZGID of each object.
[0163] Step 603: The MDS sends the ZGID of each object and the DN list corresponding to each ZGID to the SDK client.
[0164] Step 604: The SDK client initiates a request to obtain the shard data to the DN in the DN list.
[0165] For each object included in the target file, the SDK initiates a fragment data acquisition request to each DN of the object, and the fragment data acquisition request carries the ZGID.
[0166] For example, when the DN list contains the IP addresses of 5 DNs, the SDK client initiates fragment data acquisition requests to these 5 IP addresses respectively to obtain the fragment data stored in each DN.
[0167] Step 605: The DN determines the Zone associated with the ZGID.
[0168] Step 606: DN returns the shard data read from the Zone to the SDK client.
[0169] In summary, the embodiment of the present application uses ZoneGroup design, based on ZG overall scheduling and management, to support the distributed object storage system based on SMR disks to be perfectly compatible with erasure code technology. A group of data shards corresponding to EC are concurrently written in batches to the Zone space belonging to ZG, and the management node uniformly caches and manages the status of all Zones of all SMR disks in the distributed system to provide ZG reuse, file recovery, and load balancing capabilities, and ultimately realize the support of the distributed object storage system for the new generation of SMR disk storage media, solving the problem that the existing distributed object storage system cannot be directly managed based on the SMR disk Zone area, and provides higher disk space utilization than the multi-copy mode.
[0170] In addition, based on the ZGID design and the transparent transmission of ZGID between various modules of the system, the management node builds the ZG global cache to provide global ZG query, management and load capabilities. The management node generates ZGID based on the SDK client when applying for space for the first time, and finally transparently transmits it to the storage node through the SDK client, and completes the construction of the above global ZG cache based on the periodic and real-time Zone reporting of the storage node.
[0171] Combined with the actual project scale estimation, 300 storage nodes with 36 disk slots, 70,000 zones per storage node, and a total of 14TB of storage space, it is concluded that after the system has been running for a long time, the number of internal ZG caches may reach hundreds of millions. Therefore, the ZG information cached in the memory state needs to be specially partitioned and stored through hashing to avoid a single data structure storing hundreds of millions of records, which will cause serious time-consuming queries and traversals, affecting user experience and system efficiency. The management node distributes each ZGID according to the hash of fileId, so that all ZGIDs are evenly distributed to 256 sub-partitions, thereby avoiding the problem of system performance degradation caused by the huge number of ZGs in the long-term operation environment of a large number of storage nodes.
[0172] This proposal is applicable to the field of distributed object storage. By performing ZoneGroup group management on the native SMR disk Zone space and transparently transmitting the ZGID between the modules of the system, it supports the data writing capability based on the SMR disk. And through certain ZG group management, the system supports the erasure code technology and improves data security.
[0173] See also Figure 7 Based on the same inventive concept, the embodiment of the present application further provides a distributed object storage device 70, which is applied to a terminal device included in a distributed object storage system, and the device includes:
[0174] A request unit 701 is used to send a storage space application request to a management node device based on a target object to be stored, where the storage space application request carries object attribute information of the target object;
[0175] The receiving unit 702 is used to receive the storage area set identifier returned by the management node device in response to the storage space application request, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to the storage area set identifier belong; the storage area set identifier is generated by the management node device according to the object attribute information, and is used to uniquely identify the storage area set storing the target object;
[0176] The data writing unit 703 is used to send shard data writing requests to N storage node devices respectively based on the network identifiers of the N storage node devices. Each shard data writing request carries a storage area set identifier and is used to request to store a shard data of the target object in the corresponding storage node device.
[0177] Optionally, the request unit 701 is further configured to:
[0178] Based on the target file to be stored, a file creation request is sent to the management node device, where the file creation request is used to request to assign a file identifier to the target file;
[0179] Receiving a file identifier returned by the management node device in response to the file creation request, where the file identifier uniquely identifies the target file;
[0180] Based on the target object included in the target file, a storage space application request is sent to the management node device, and the object attribute information carried in the storage space application request includes a file identifier and an object identifier of the target object.
[0181] Optionally, the request unit 701 is further configured to:
[0182] Based on the target data type to be stored, a storage space creation request is sent to the management node device, where the storage space creation request is used to request allocation of a storage space identifier for itself;
[0183] Receiving a storage space identifier returned by the management node device in response to the storage space creation request, where the storage space identifier uniquely identifies data of a target data type output by the terminal device;
[0184] Optionally, the device further includes a file reading unit 704;
[0185] The request unit 701 is further used to: send a file retrieval request to the management node device based on the name of the target file to be read;
[0186] The receiving unit 702 is further configured to receive the storage area set identifiers of each object returned by the management node device in response to the file retrieval request, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to each storage area set identifier belong;
[0187] The file reading unit 704 is used to send a shard data acquisition request to N storage node devices corresponding to each object in each object, where the shard data acquisition request carries a storage area set identifier of the object; and
[0188] The shard data returned by each storage node device in response to the corresponding shard data acquisition request is received, and the corresponding object data is obtained based on the shard data of each object to obtain a target file composed of the object data of each object.
[0189] Optionally, the storage space application request also carries a data encoding type used to store the target object, and the number N of storage node devices is determined according to the data encoding type;
[0190] The file writing unit 703 is specifically used for:
[0191] Based on the data encoding type, encode the target object to obtain N slice data of the target object;
[0192] Based on the network identifiers of the N storage node devices, the obtained N shard data are respectively carried in shard data write requests and sent to corresponding storage node devices.
[0193] The device can be used to perform Figure 3 to Figure 6 The method executed by the SDK client in the embodiment shown in FIG. 1 is shown in FIG. 1 . Therefore, for the functions that can be realized by each functional module of the device, reference can be made to FIG. Figure 3 to Figure 6 The description of the illustrated embodiment is not repeated in detail.
[0194] See also Figure 8 Based on the same inventive concept, the embodiment of the present application further provides a distributed object storage device 80, which is applied to a management node device included in a distributed object storage system. The device includes:
[0195] An identifier generating unit 801 is used to generate a storage area set identifier of a target object to be stored based on the object attribute information carried in the storage space application request sent by the terminal device, where the storage area set identifier uniquely identifies a storage area set storing the target object;
[0196] The storage node management unit 802 is used to select N storage node devices from a plurality of storage node devices included in the distributed object storage system based on a preset load balancing strategy, wherein the N target storage areas corresponding to the storage area set identifiers are derived from the N storage node devices, and one storage area is derived from one storage node device;
[0197] The transceiver unit 803 is used to return the storage area set identifier and the network identifiers of the N storage node devices to the terminal device; and,
[0198] Receive storage area attribute information reported by N storage node devices respectively, and update the storage mapping relationship stored in itself based on the storage area attribute information, the storage area attribute information is used to indicate the storage area identifier associated with the storage area set identifier, and the storage mapping relationship is used to indicate the storage area attribute information corresponding to each storage area set identifier.
[0199] Optionally, the identifier generating unit 801 is further configured to:
[0200] In response to a file creation request sent by a terminal device, a file identifier is assigned to a target file, the file identifier uniquely identifies the target file; the target file includes a target object;
[0201] A storage area set identifier is generated based on the file identifier and the object identifier of the target object.
[0202] Optionally, the device further includes a retrieval unit 804, configured to:
[0203] Based on the target file name carried in the file retrieval request sent by the terminal device, query the storage area set identifiers of each object included in the target file in the storage mapping relationship, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to each storage area set identifier belong;
[0204] The storage area set identifiers of each object and the network identifiers of the N storage node devices corresponding to each storage area set identifier are sent to the terminal device.
[0205] Optionally, the storage mapping relationship is cached in the memory of the management node device, and the storage mapping relationship is stored in multiple sub-tables, each sub-table storing a part of the mapping relationship;
[0206] The retrieval unit 804 is specifically used for:
[0207] A parallel processing method is adopted to perform traversal query on multiple subtables respectively to obtain the storage area set identifier of each object.
[0208] The device can be used to perform Figure 3 to Figure 6 The method executed by the MDS in the embodiment shown in FIG. 1 is shown in FIG. 1 . Therefore, for the functions that can be implemented by each functional module of the device, reference can be made to FIG. Figure 3 to Figure 6 The description of the illustrated embodiment is not repeated in detail.
[0209] See also Fig. 9 Based on the same inventive concept, the embodiment of the present application further provides a distributed object storage device 90, which is applied to a storage node device included in a distributed object storage system. The device includes:
[0210] The receiving unit 901 is used to receive a shard data write request sent by a terminal device, where the shard data write request carries a storage area set identifier of a target object to be stored, and the shard data write request carries a shard data of the target object;
[0211] The storage area management unit 902 is used to select a target storage area from the storage disks included in the storage area based on the shard data write request and the usage of the storage disks included in the storage area, and store the shard data in the target storage area; and associate the storage area set identifier with the target storage area;
[0212] The sending unit 903 is used to report storage area attribute information to the management node device, where the storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the storage area set identifier in the storage node device.
[0213] Optionally, the storage disk is a shingled magnetic recording SMR disk.
[0214] The device can be used to perform Figure 3 to Figure 6 The method executed by the DN in the embodiment shown in FIG. 1 is a method executed by the DN in the embodiment shown in FIG. 1 . Therefore, for the functions that can be realized by each functional module of the device, reference can be made to FIG. Figure 3 to Figure 6 The description of the illustrated embodiment is not repeated in detail.
[0215] See also Fig.10 Based on the same technical concept, an embodiment of the present application also provides a computer device 100, which may include a memory 1001 and a processor 1002.
[0216] The memory 1001 is used to store computer programs executed by the processor 1002. The memory 1001 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the computer device, etc. The processor 1002 may be a central processing unit (CPU), or a digital processing unit, etc. The specific connection medium between the above-mentioned memory 1001 and the processor 1002 is not limited in the embodiments of the present application. The embodiments of the present application are Fig.10 In the embodiment, the memory 1001 and the processor 1002 are connected via a bus 1003. The bus 1003 is connected to the processor 1002 via a bus 1003. Fig.10 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 1003 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0217] The memory 1001 may be a volatile memory, such as a random-access memory (RAM); the memory 1001 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 1001 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1001 may be a combination of the above memories.
[0218] The processor 1002 is used to execute the following when calling the computer program stored in the memory 1001: Figure 3 to Figure 6 The method executed by the SDK client, MDS or DN in the illustrated embodiment.
[0219] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the following steps: Figure 3 to Figure 6 The method executed by the SDK client, MDS or DN in the illustrated embodiment.
[0220] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0221] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0222] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A distributed object storage method, It is characterized in that Applied to a management node device included in a distributed object storage system, the method includes: Generate a storage area set identifier of a target object to be stored based on the object attribute information carried in the storage space application request sent by the terminal device, wherein the object attribute information is used to uniquely identify the target object, and the storage area set identifier is used to uniquely identify a storage area set storing the target object; Based on a preset load balancing strategy, N storage node devices are selected from a plurality of storage node devices included in the distributed object storage system, wherein the N target storage areas corresponding to the storage area set identifiers are derived from the N storage node devices, and one storage area is derived from one storage node device, each of the N storage node devices includes a plurality of storage areas, and the storage disk included in each storage node device is a shingled magnetic recording SMR disk; Returning the storage area set identifier and the network identifiers of the N storage node devices to the terminal device; Receive the storage area attribute information reported by each of the N storage node devices, and update the storage mapping relationship stored in its own storage based on the storage area attribute information, the storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the storage area set identifier in the corresponding storage node device, and the storage area attribute information is reported by the corresponding storage node device after receiving the shard data write request sent by the terminal device, and the storage mapping relationship is used to indicate the storage area attribute information corresponding to each storage area set identifier.
2. The method according to claim 1, It is characterized in that Before generating a storage area set identifier of a target object to be stored based on the object attribute information carried in the storage space application request sent by the terminal device, the method further includes: In response to the file creation request sent by the terminal device, a file identifier is assigned to a target file, wherein the file identifier uniquely identifies the target file; the target file includes the target object; The object attribute information includes the file identifier and the object identifier of the target object. Based on the object attribute information carried in the storage space application request sent by the terminal device, a storage area set identifier of the target object to be stored is generated, including: The storage area set identifier is generated based on the file identifier and the object identifier of the target object.
3. The method according to claim 1, It is characterized in that The method further comprises: Based on the target file name carried in the file retrieval request sent by the terminal device, querying the storage area set identifiers of each object included in the target file in the storage mapping relationship, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to each storage area set identifier belong; The storage area set identifiers of the respective objects and the network identifiers of the N storage node devices corresponding to each storage area set identifier are sent to the terminal device.
4. The method according to claim 3, It is characterized in that The storage mapping relationship is cached in the memory of the management node device, and the storage mapping relationship is stored in a plurality of sub-tables, each sub-table storing a portion of the storage mapping relationship; Based on the target file name carried in the file retrieval request sent by the terminal device, the storage area set identifiers of each object included in the target file are searched in the storage mapping relationship, including: The plurality of subtables are traversed and queried respectively in a parallel processing manner to obtain storage area set identifiers of the respective objects.
5. A distributed object storage method, It is characterized in that Applied to a terminal device included in a distributed object storage system, the method includes: Based on the target object to be stored, sending a storage space application request to the management node device, the storage space application request carrying object attribute information of the target object, the object attribute information being used to uniquely identify the target object; Receive the storage area set identifier returned by the management node device in response to the storage space application request, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to the storage area set identifier belong; the storage area set identifier is generated by the management node device according to the object attribute information, and is used to uniquely identify the storage area set storing the target object, the N storage node devices are selected by the management node device from the multiple storage node devices included in the distributed object storage system based on a preset load balancing strategy, one storage area originates from one storage node device, each of the N storage node devices includes multiple storage areas, and the storage disk included in each storage node device is an SMR disk; Based on the network identifiers of the N storage node devices, shard data write requests are sent to the N storage node devices respectively, each shard data write request carries the storage area set identifier and is used to request to store a shard data of the target object in the corresponding storage node device, wherein after receiving the shard data write request, the N storage node devices associate the storage area set identifier with the target storage area included in themselves, and report storage area attribute information to the management node device, wherein the storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the storage area set identifier in the corresponding storage node device.
6. The method according to claim 5, It is characterized in that Before sending a storage space application request to a management node device based on the target object to be stored, the method further includes: Based on the target file to be stored, sending a file creation request to the management node device, wherein the file creation request is used to request to assign a file identifier to the target file; Receiving a file identifier returned by the management node device in response to the file creation request, wherein the file identifier uniquely identifies the target file; Then, based on the target object to be stored, a storage space application request is sent to the management node device, and the storage space application request carries the object attribute information of the target object, including: Based on the target object included in the target file, the storage space application request is sent to the management node device, and the object attribute information carried by the storage space application request includes the file identifier and the object identifier of the target object.
7. The method according to claim 5, It is characterized in that Before sending a storage space application request to a management node device based on the target object to be stored, the method further includes: Based on the target data type to be stored, sending a storage space creation request to the management node device, wherein the storage space creation request is used to request allocation of a storage space identifier for itself; A storage space identifier is received which is returned by the management node device in response to the storage space creation request, wherein the storage space identifier uniquely identifies data of a target data type output by the terminal device.
8. The method according to claim 5, It is characterized in that The method further comprises: Based on the name of the target file to be read, sending a file retrieval request to the management node device; Receiving the storage area set identifiers of each object returned by the management node device in response to the file retrieval request, and the network identifiers of the N storage node devices to which the N target storage areas corresponding to each storage area set identifier belong; For each of the objects, a shard data acquisition request is sent to the N storage node devices corresponding to the object, wherein the shard data acquisition request carries a storage area set identifier of the object; The shard data returned by each storage node device in response to the corresponding shard data acquisition request is received, and the corresponding object data is obtained based on the shard data of each object to obtain a target file composed of the object data of each object.
9. The method according to any one of claims 5 to 8, It is characterized in that The storage space application request also carries a data encoding type used to store the target object, and the number N of storage node devices is determined according to the data encoding type; Then, based on the network identifiers of the N storage node devices, shard data write requests are sent to the N storage node devices respectively, including: Based on the data encoding type, encode the target object to obtain N slice data of the target object; Based on the network identifiers of the N storage node devices, the obtained N shard data are respectively carried in shard data write requests and sent to corresponding storage node devices.
10. A distributed object storage method, It is characterized in that Applied to a storage node device included in a distributed object storage system, the method includes: Receive a shard data write request sent by a terminal device, the shard data write request is sent by the terminal device after receiving a storage area set identifier and a network identifier of N storage node devices sent by a management node device included in the distributed object storage system, the shard data write request carries the storage area set identifier of a target object to be stored, and the shard data write request carries a shard data of the target object; wherein the storage area set identifier is generated by the management node device according to object attribute information, and is used to uniquely identify a storage area set storing the target object, the object attribute information is used to uniquely identify the target object, the N target storage areas included in the storage area set are derived from the N storage node devices, the N storage node devices are selected by the management node device from a plurality of storage node devices included in the distributed object storage system based on a preset load balancing strategy, and one storage area is derived from one storage node device, and each of the N storage node devices includes a plurality of storage areas; Based on the slice data write request and the usage of the storage disk included in the storage disk, a target storage area is selected from the storage disk included in the storage disk, and the slice data is stored in the target storage area, where the storage disk is an SMR disk; and Associating the storage area set identifier with the target storage area; Reporting storage area attribute information to a management node device, wherein the storage area attribute information is used to indicate a storage area identifier of the target storage area associated with the storage area set identifier in the storage node device.
11. A distributed object storage device, It is characterized in that Applied to a terminal device included in a distributed object storage system, the device includes: A request unit, configured to send a storage space application request to a management node device based on a target object to be stored, wherein the storage space application request carries object attribute information of the target object, and the object attribute information is used to uniquely identify the target object; A receiving unit, configured to receive a storage area set identifier returned by the management node device in response to the storage space application request, and network identifiers of N storage node devices to which N target storage areas corresponding to the storage area set identifier belong; the storage area set identifier is generated by the management node device according to the object attribute information, and is used to uniquely identify a storage area set storing the target object; the N storage node devices are selected by the management node device from a plurality of storage node devices included in the distributed object storage system based on a preset load balancing strategy; one storage area originates from one storage node device; each of the N storage node devices includes a plurality of storage areas; and the storage disk included in each storage node device is an SMR disk; A data writing unit is used to send shard data writing requests to the N storage node devices respectively based on the network identifiers of the N storage node devices, each shard data writing request carries the storage area set identifier and is used to request to store a shard data of the target object in the corresponding storage node device, wherein after receiving the shard data writing request, the N storage node devices associate the storage area set identifier with the target storage area included in themselves, and report storage area attribute information to the management node device, wherein the storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the storage area set identifier in the corresponding storage node device.
12. A distributed object storage device, It is characterized in that Applied to a management node device included in a distributed object storage system, the device includes: an identification generating unit, configured to generate a storage area set identification of a target object to be stored based on object attribute information carried in a storage space application request sent by a terminal device, wherein the object attribute information is used to uniquely identify the target object, and the storage area set identification is used to uniquely identify a storage area set storing the target object; A storage node management unit, configured to select N storage node devices from a plurality of storage node devices included in the distributed object storage system based on a preset load balancing strategy, wherein the N target storage areas corresponding to the storage area set identifiers originate from the N storage node devices, and one storage area originates from one storage node device, each of the N storage node devices includes a plurality of storage areas, and the storage disk included in each storage node device is an SMR disk; a transceiver unit, configured to return the storage area set identifier and the network identifiers of the N storage node devices to the terminal device; and Receive the storage area attribute information reported by each of the N storage node devices, and update the storage mapping relationship stored in its own storage based on the storage area attribute information, the storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the storage area set identifier in the corresponding storage node device, and the storage area attribute information is reported by the corresponding storage node device after receiving the shard data write request sent by the terminal device, and the storage mapping relationship is used to indicate the storage area attribute information corresponding to each storage area set identifier.
13. A distributed object storage device, It is characterized in that Applied to a storage node device included in a distributed object storage system, the device includes: A receiving unit, configured to receive a shard data write request sent by a terminal device, wherein the shard data write request is sent by the terminal device after receiving a storage area set identifier and a network identifier of N storage node devices sent by a management node device included in a distributed object storage system, wherein the shard data write request carries the storage area set identifier of a target object to be stored, and the shard data write request carries one shard data of the target object; wherein the storage area set identifier is generated by the management node device according to object attribute information, and is used to uniquely identify a storage area set storing the target object, and the object attribute information is used to uniquely identify the target object, the N target storage areas included in the storage area set are derived from the N storage node devices, and the N storage node devices are selected by the management node device from a plurality of storage node devices included in the distributed object storage system based on a preset load balancing strategy, and one storage area is derived from one storage node device, and each storage node device in the N storage node devices includes a plurality of storage areas; A storage area management unit, configured to select a target storage area from the storage disks included in the storage area based on the shard data write request and the usage of the storage disks included in the storage area, and store the shard data in the target storage area, wherein the storage disk is an SMR disk; and associate the storage area set identifier with the target storage area; The sending unit is used to report storage area attribute information to the management node device, wherein the storage area attribute information is used to indicate the storage area identifier of the target storage area associated with the storage area set identifier in the storage node device.
14. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4, 5 to 9 or 10 are implemented.
15. A computer storage medium having computer program instructions stored thereon, It is characterized in that When the computer program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 4, 5 to 9 or 10 are implemented.
Citation Information
Patent Citations
Video data storage method and system
CN105404469A
Data storage method and data storage system
CN107291889A
Object storage method and device
CN111352577A
Distributed data dynamic storage method based on block chain and electronic equipment
CN112182112A
Data processing method and device, electronic equipment and storage medium
CN112988683A
