A data processing method, apparatus, device, and medium

CN115469806BActive Publication Date: 2026-09-15ZHEJIANG DAHUA TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211077051.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-05
Publication Date
2026-09-15
Estimated Expiration
2042-09-05

AI Technical Summary

Technical Problem

[0005]本申请实施例提供了一种数据处理方法、装置、设备及介质,用于解决叠瓦式磁记录盘数据处理操作效率较低的问题

Benefits of technology

[0072] In this embodiment, after acquiring the first target data, a first data block is allocated in the data subspace of the storage space based on the data length of the first target data. Since the size of the first data block is not limited when allocating it, first target data of different lengths can be allocated first data blocks of different sizes, avoiding waste of data block space and improving storage flexibility. After allocating the first data block, a first data region is further allocated to it, which allows the first data block to be stored in a suitable first data region without wasting space and reduces the cost of space reclamation after deletion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115469806B_ABST
    Figure CN115469806B_ABST
Patent Text Reader

Abstract

The application relates to the technical field of data storage, in particular to a data processing method and device, equipment and medium. The application can be used in data storage systems and the like, and solves the problem of low data processing efficiency. The method comprises the following steps: obtaining first target data; based on the data length of the first target data, allocating a first data block for the first target data in a data subspace of a storage space, and based on the size of the first data block, allocating a first data area for the first data block, obtaining data block attribute information of the first data block and area attribute information of the first data area; based on the data block attribute information and the area attribute information, determining corresponding first storage directories in a plurality of candidate storage directories contained in a file subspace of the storage space, respectively storing the data block attribute information and the area attribute information in the corresponding first storage directories; writing the first target data into the first data block, and storing the first data block in the first data area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a data processing method, apparatus, device and medium. Background Technology

[0002] With the continuous development of technology, more and more devices can apply shingled magnetic recording (SMR) to distributed object storage systems, which can reduce user costs while obtaining greater storage capacity.

[0003] In related technologies, because SMR disks use track overlap technology, when writing target data to an SMR disk, in order to ensure the integrity of data on adjacent tracks, the data on adjacent tracks needs to be read out before writing, reorganized together with the target data, and then written sequentially. This makes the data writing process relatively complex, resulting in low data writing efficiency. When reading target data from an SMR disk, because a single SMR disk stores a large number of data blocks, and all data is stored in the same directory, reading the target data from this directory may require traversing all data to retrieve the target data, resulting in low data read and access efficiency.

[0004] It is evident that the efficiency of data processing is relatively low when applying SMR disks to distributed object storage systems. Summary of the Invention

[0005] This application provides a data processing method, apparatus, device, and medium to solve the problem of low data processing efficiency of shingled magnetic recording disks.

[0006] Firstly, a data processing method is provided, including:

[0007] In response to a write operation on the first target data, the first target data is obtained;

[0008] Based on the data length of the first target data, a first data block is allocated for the first target data in the data subspace of the storage space, and a first data region is allocated for the first data block based on the size of the first data block. The data block attribute information of the first data block and the region attribute information of the first data region are obtained respectively.

[0009] Based on the data block attribute information and the region attribute information, a corresponding first storage directory is determined from the multiple alternative storage directories contained in the file subspace of the storage space, and the data block attribute information and the region attribute information are stored in the corresponding first storage directory.

[0010] Based on the data block attribute information and the region attribute information, the first target data is written into the first data block, and the first data block is stored in the first data region.

[0011] Optionally, the data block attribute information includes the data block number of the first data block, which is used to uniquely identify the first data block, including:

[0012] Based on the number of directories in the multiple alternative storage directories, the data block numbers are converted to obtain the data block storage location;

[0013] Based on the data block storage location, a first storage directory for the data block attribute information is determined from the plurality of alternative storage directories;

[0014] Based on the region attribute information and the multiple alternative storage directories, a first storage directory for the region attribute information is determined.

[0015] The process of performing data conversion on the data block number to obtain the data block storage location is to distribute the data block attribute information across multiple directories, which can prevent the slow access speed of attribute information caused by storing too many files in a single directory.

[0016] Optionally, the file subspace includes a region pointer, which indicates the region storage location of the region attribute information to be stored, including:

[0017] When the region storage location indicated by the region pointer in the file subspace is determined to be a candidate storage directory, the candidate storage directory is used as the first storage directory;

[0018] When it is determined that the region storage location indicated by the region pointer is a location other than the plurality of alternative storage directories, the first storage directory is created at the region storage location.

[0019] Optionally, after writing the first target data into the first data block and storing the first data block in the first data region based on the data block attribute information and the region attribute information, the method further includes:

[0020] In response to a retrieval operation targeting the second target data, the identification information of the second target data is obtained;

[0021] Based on the identification information of the second target data, in the file subspace, the second storage directories corresponding to the second data block and the second data area are determined respectively, and the data block attribute information of the second data block and the data area attribute information of the second data area are read from the obtained second storage directories respectively, wherein the second data block is used to store the second target data, and the second data area is used to store the second data block;

[0022] Based on the data block attribute information of the second data block and the data region attribute information of the second data region, the second target data is read in the data subspace.

[0023] Optionally, the identification information includes a data block number of the second data block, the data block number being used to uniquely identify the second data block, including:

[0024] Based on the number of candidate storage directories contained in the file subspace, the data block number is converted to obtain the second storage directory of the second data block;

[0025] Read the data block attribute information of the second data block from the second storage directory of the second data block;

[0026] Based on the data block attribute information, determine the second storage directory of the second data area;

[0027] Read the region attribute information of the second data region from the second storage directory of the second data region.

[0028] Optionally, the identification information includes the region offset position of the second data region, which represents the offset of the storage location of the region attribute information of the second data region relative to the region logical address. The region logical address is the starting address of the data subspace and includes:

[0029] The offset position of the region is rounded up according to the number of candidate storage directories contained in the file subspace to obtain the second storage directory of the second data region;

[0030] Read the region attribute information of the second data region from the second storage directory of the second data region;

[0031] Based on the region attribute information, determine the second storage directory of the second data block;

[0032] Read the data block attribute information of the second data block from the second storage directory of the second data block.

[0033] Optionally, the region attribute information includes, after the data block description information of each data block stored in the corresponding data region within the data subspace, the following:

[0034] In response to a deletion operation on the third target data, the data block number of the third data block is obtained, and in the file subspace, the third storage directory corresponding to the third data block and the third data area is determined, wherein the third data block is used to store the third target data, and the third data area is used to store the third data block;

[0035] Delete the data block attribute information of the third data block from the third storage directory corresponding to the third data block;

[0036] In the third storage directory corresponding to the third data region, delete the data block description information of the third data block contained in the region attribute information of the third data region.

[0037] Secondly, a data processing apparatus is provided, comprising:

[0038] Acquisition module: used to acquire the first target data in response to a write operation on the first target data;

[0039] Processing module: Based on the data length of the first target data, allocate a first data block for the first target data in the data subspace of the storage space, allocate a first data region for the first data block, and obtain the data block attribute information of the first data block and the region attribute information of the first data region respectively;

[0040] The processing module is further configured to: based on the data block attribute information and the region attribute information, determine the corresponding first storage directory from among the multiple alternative storage directories contained in the file subspace of the storage space, and store the data block attribute information and the region attribute information into the corresponding first storage directory respectively;

[0041] The processing module is further configured to: write the first target data into the first data block based on the data block attribute information and the region attribute information, and store the first data block into the first data region.

[0042] Optionally, the data block attribute information includes the data block number of the first data block, the data block number being used to uniquely identify the first data block, and the processing module is specifically used for:

[0043] Based on the number of directories in the multiple alternative storage directories, the data block numbers are converted to obtain the data block storage location;

[0044] Based on the data block storage location, a first storage directory for the data block attribute information is determined from the plurality of alternative storage directories;

[0045] Based on the region attribute information and the multiple alternative storage directories, a first storage directory for the region attribute information is determined.

[0046] Optionally, the file subspace includes a region pointer, which indicates the region storage location of the region attribute information to be stored. The processing module is specifically used for:

[0047] When the region storage location indicated by the region pointer in the file subspace is determined to be a candidate storage directory, the candidate storage directory is used as the first storage directory;

[0048] When it is determined that the region storage location indicated by the region pointer is a location other than the plurality of alternative storage directories, the first storage directory is created at the region storage location.

[0049] Optionally, the processing module is further configured to:

[0050] After writing the first target data into the first data block based on the data block attribute information and the region attribute information, and storing the first data block in the first data region, in response to a retrieval operation for the second target data, the identification information of the second target data is obtained;

[0051] Based on the identification information of the second target data, in the file subspace, the second storage directories corresponding to the second data block and the second data area are determined respectively, and the data block attribute information of the second data block and the data area attribute information of the second data area are read from the obtained second storage directories respectively, wherein the second data block is used to store the second target data, and the second data area is used to store the second data block;

[0052] Based on the data block attribute information of the second data block and the data region attribute information of the second data region, the second target data is read in the data subspace.

[0053] Optionally, the identification information includes the data block number of the second data block, the data block number being used to uniquely identify the second data block, and the processing module is specifically used for:

[0054] Based on the number of candidate storage directories contained in the file subspace, the data block number is converted to obtain the second storage directory of the second data block;

[0055] Read the data block attribute information of the second data block from the second storage directory of the second data block;

[0056] Based on the data block attribute information, determine the second storage directory of the second data area;

[0057] Read the region attribute information of the second data region from the second storage directory of the second data region.

[0058] Optionally, the identification information includes the region offset position of the second data region, the region offset position representing the offset of the storage location of the region attribute information of the second data region relative to the region logical address, the region logical address being the starting address of the data subspace, and the processing module is specifically used for:

[0059] The offset position of the region is rounded up according to the number of candidate storage directories contained in the file subspace to obtain the second storage directory of the second data region;

[0060] Read the region attribute information of the second data region from the second storage directory of the second data region;

[0061] Based on the region attribute information, determine the second storage directory of the second data block;

[0062] Read the data block attribute information of the second data block from the second storage directory of the second data block.

[0063] Optionally, the region attribute information includes data block description information of each data block stored in the corresponding data region within the data subspace. The processing module is further configured to:

[0064] Based on the data block attribute information and the region attribute information, the first target data is written into the first data block and the first data block is stored in the first data region. In response to the deletion operation for the third target data, the data block number of the third data block is obtained. In the file subspace, the third storage directory corresponding to the third data block and the third data region is determined. The third data block is used to store the third target data, and the third data region is used to store the third data block.

[0065] Delete the data block attribute information of the third data block from the third storage directory corresponding to the third data block;

[0066] In the third storage directory corresponding to the third data region, delete the data block description information of the third data block contained in the region attribute information of the third data region.

[0067] Thirdly, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0068] Fourthly, a computer device is provided, comprising:

[0069] Memory, used to store program instructions;

[0070] A processor is configured to invoke program instructions stored in the memory and execute the method described in the first aspect according to the obtained program instructions.

[0071] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method as described in the first aspect.

[0072] In this embodiment, after acquiring the first target data, a first data block is allocated in the data subspace of the storage space based on the data length of the first target data. Since the size of the first data block is not limited when allocating it, first target data of different lengths can be allocated first data blocks of different sizes, avoiding waste of data block space and improving storage flexibility. After allocating the first data block, a first data region is further allocated to it, which allows the first data block to be stored in a suitable first data region without wasting space and reduces the cost of space reclamation after deletion.

[0073] Furthermore, based on the data block attribute information and region attribute information, appropriate storage directories are selected from multiple candidate storage directories in the file subspace of the storage space, and the data block attribute information and region attribute information are stored in the corresponding directories. This allows different data block attribute information and region attribute information to be stored in different directories. Each time data is retrieved, only a small amount of data stored in the corresponding directory needs to be queried, without having to traverse all the data stored in the file subspace, which makes subsequent data maintenance work such as regular backups and checks more convenient.

[0074] Furthermore, based on the data block attribute information and the region attribute information, the target data is written into the allocated data block, and the data block is stored in the data region. Each data block corresponds to a data block attribute information, and each region storing a data block corresponds to a region attribute information. Similar to a standard file system, when writing data, the target data can be directly written into the data block, avoiding the problem of low writing efficiency caused by reading data from adjacent tracks, reorganizing it with the target data, and then writing it sequentially. This makes the write operation simpler and more efficient. Attached Figure Description

[0075] Figure 1A This is a schematic diagram of a file storage method in the prior art;

[0076] Figure 1B This is one application scenario of the data processing method provided in the embodiments of this application;

[0077] Figure 2 A schematic flowchart of a data processing method provided in an embodiment of this application is shown below;

[0078] Figure 3 A schematic diagram illustrating the principle of a data processing method provided in this application embodiment;

[0079] Figure 4A A schematic diagram of the data processing method provided in the embodiments of this application. Figure 2 ;

[0080] Figure 4B A schematic diagram of the data processing method provided in the embodiments of this application. Figure 2 ;

[0081] Figure 5 A flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 ;

[0082] Figure 6 A flowchart illustrating a data processing method provided in an embodiment of this application. Figure 3 ;

[0083] Figure 7 A schematic diagram of the data processing method provided in the embodiments of this application. Figure 3 ;

[0084] Figure 8 A schematic diagram of the structure of a data processing apparatus provided in the embodiments of this application is shown below;

[0085] Figure 9 A schematic diagram of the structure of a data processing apparatus provided in the embodiments of this application. Figure 2 . Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0087] It should be noted that in the embodiments of this application, the acquisition of first target data and other related data is involved. When the above embodiments of this application are applied to specific products or technologies, each acquisition of data requires user permission or consent, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0088] The application areas of the data processing method provided in the embodiments of this application will be briefly introduced below.

[0089] With the continuous development of technology, more and more devices can apply shingled magnetic recording (SMR) to distributed object storage systems, which can reduce user costs while obtaining greater storage capacity.

[0090] In related technologies, because SMR disks use track overlap technology, when writing target data to an SMR disk, in order to ensure the integrity of data on adjacent tracks, the data on adjacent tracks needs to be read out before writing, reorganized together with the target data, and then written sequentially. This makes the data writing process relatively complex, resulting in low data writing efficiency. When reading target data from an SMR disk, because a single SMR disk stores a large number of data blocks, and all data is stored in the same directory, reading the target data from this directory may require traversing all data to retrieve the target data, resulting in low data read and access efficiency.

[0091] For example, please refer to Figure 1A In the schematic diagram of SMR disk track overlap technology, the SMR disk includes a read head 11 and a write head 12. Adjacent tracks have overlapping portions. During data writing operations, to prevent newly written data from affecting data already stored on adjacent tracks, all data on adjacent tracks needs to be read out, reorganized with the data to be written, and then written sequentially. For example, if tracks N and N+1 are adjacent tracks, when writing data to track N, all data already stored on track N+1 needs to be read out, reorganized with the data to be written to track N, and then written sequentially to tracks N and N+1.

[0092] It is evident that the efficiency of data processing is relatively low when applying SMR disks to distributed object storage systems.

[0093] In this embodiment, after acquiring the first target data, a first data block is allocated in the data subspace of the storage space based on the data length of the first target data. Since the size of the first data block is not limited when allocating it, first target data of different lengths can be allocated first data blocks of different sizes, avoiding waste of data block space and improving storage flexibility. After allocating the first data block, a first data region is further allocated to it, which allows the first data block to be stored in a suitable first data region without wasting space and reduces the cost of space reclamation after deletion.

[0094] Furthermore, based on data block attribute information and region attribute information, appropriate storage directories are selected from multiple alternative storage directories in the file subspace of the storage space, and the data block attribute information and region attribute information are stored in the corresponding directories. This allows different data block attribute information and region attribute information to be stored in different directories. Each time data is retrieved, only a small amount of data stored in the corresponding directory needs to be queried, without having to traverse all the data stored in the file subspace. This makes subsequent data maintenance work such as regular backups and checks more convenient.

[0095] Furthermore, based on data block attribute information and region attribute information, the target data is written to the allocated data block, and the data block is stored in the data region. Each data block corresponds to a data block attribute information, and each region storing the data block corresponds to a region attribute information. Similar to the standard file system, when writing data, the target data can be directly written to the data block. This avoids the problem of low writing efficiency caused by reading data from adjacent tracks, reorganizing it with the target data, and then writing it sequentially. This makes the write operation simpler and more efficient.

[0096] The following describes the application scenarios of the data processing method provided in this application.

[0097] Please refer to Figure 1B This is a schematic diagram illustrating an application scenario of the data processing method provided in this application. The application scenario includes a client 101 and a server 102. The client 101 and the server 102 can communicate with each other. The communication method can be wired, such as through a network cable or serial cable; or wireless, such as through Bluetooth or Wi-Fi. No specific limitation is imposed.

[0098] Client 101 generally refers to a device that can provide instructions or data to be processed to server 102, such as a terminal device, a third-party application accessible to the terminal device, or a webpage accessible to the terminal device. Terminal devices include, but are not limited to, mobile phones, computers, intelligent transportation equipment, and smart appliances. Server 102 generally refers to a device that can process data, such as a terminal device or a server. Servers include, but are not limited to, cloud servers, local servers, or associated third-party servers. Both client 101 and server 102 can use cloud computing to reduce the consumption of local computing resources; similarly, they can also use cloud storage to reduce the consumption of local storage resources.

[0099] As one embodiment, the client 101 and the server 102 can be the same device. For example, this embodiment can be implemented using only the client 101 or only the server 102, etc., without any specific limitation. In this embodiment, the client 101 and the server 102 are described as different devices.

[0100] The following is based on Figure 1B Taking server 102 as an example, the data processing method provided in this application embodiment will be described in detail. Please refer to... Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of this application.

[0101] S201, in response to a write operation on the first target data, obtain the first target data.

[0102] Possible ways to obtain the first target data include: the user triggers a write operation through the client, the client obtains the first target data, the client sends the first target data to the server, and the server receives the first target data sent by the client; it also includes other devices sending the first target data to the server, and the server receiving the first target data sent by other devices, etc., without any specific restrictions.

[0103] S202, based on the data length of the first target data, allocate a first data block for the first target data in the data subspace of the storage space, and allocate a first data region for the first data block based on the size of the first data block, and obtain the data block attribute information of the first data block and the region attribute information of the first data region respectively.

[0104] In this embodiment, an SMR disk is used as an example for the storage space. The data subspace of the storage space is the SMR region of the SMR disk. The SMR region includes multiple storage sub-regions (zones), and each storage sub-region contains multiple data blocks. Data blocks can be obtained by dividing the corresponding storage sub-region (zone). The size of a zone can be 256MB, which can be used to store various blocks, and blocks can be used to store data. The data subspace can be managed in units of zones. Within a zone, sequential writes, random reads, and overall deletions are supported, without specific limitations.

[0105] Because the length of the acquired first target data varies, the size of the first data block allocated for the first target data in the data subspace of the storage space also varies. For example, if the acquired first target data is file data that the user needs to store, and the first target data is long, then a larger first data block will be allocated. Then, based on the size of the acquired first data block, a first data region is allocated. For example, 256MB of data might be stored in two 128MB blocks, and then appropriate zones are allocated to these two blocks.

[0106] When allocating the first data region for the first data block, it's necessary to consider that if a data region is full, the next data region can be matched for writing. Since there's no limit to the block size, when writing a block to a zone, a situation may arise where the zone is full, but the block still has unwritten space. In this case, the next zone where a block can be written to is matched for writing. For example, please refer to... Figure 3 The system contains four data blocks to be allocated. These four data blocks are written sequentially within a zone. Data block A is written to zone1, which is not yet full. Then, data block B is written to zone1. Once zone1 is full, no more data blocks are written. Data block C is written to zone2, which is also not full. Data block D is relatively small, and the remaining space in zone2 is sufficient for its size, so data block D is written to zone2. Data block E is relatively large, and there is insufficient space in zone2, so it is written to zone3.

[0107] S203, based on the data block attribute information and the region attribute information, determine the corresponding first storage directory from the multiple alternative storage directories contained in the file subspace of the storage space, and store the data block attribute information and the region attribute information into the corresponding first storage directory.

[0108] The server obtains the first target data input by the user and allocates it to the first target data block. After allocating the first target data block to the corresponding first data area, it determines the appropriate first storage directory from multiple alternative storage directories contained in the file subspace of the storage space. That is, the block metadata and zone metadata are stored in the corresponding storage directory.

[0109] Continuing with the example of an SMR disk as the storage space, the file subspace of the storage space is the CMR region of the SMR disk. This CMR region can support random read and write operations just like a traditional disk. The storage space of this CMR region can be set to a relatively small size to avoid consuming excessive storage resources. The server can also format the CMR region into a standard file system to store data block attribute information, i.e., block metadata information, and zone attribute information, i.e., zone metadata information.

[0110] As one embodiment, the data block attribute information includes the data block number of the first data block. The data block number is used to uniquely identify the first data block. For example, the data block number can be a blockid, which is a unique blockid corresponding to each block.

[0111] When determining the primary storage directory for data block attribute information, the data block number can be transformed according to the number of directories in multiple candidate storage directories to obtain the data block storage location. Based on the data block storage location, the primary storage directory for data block attribute information is determined from multiple candidate storage directories. Based on region attribute information and multiple candidate storage directories, the primary storage directory for region attribute information is determined.

[0112] The number of directories in multiple alternative storage directories can be the number of directories in each alternative storage directory used to store data block attribute information in the file subspace, or it can be the total number of directories in all alternative storage directories in the file subspace; there is no specific limit.

[0113] Data transformation processing can include string conversion of data block numbers, rounding, or hashing using hash functions, etc., with no specific restrictions.

[0114] Continuing with the example of an SMR disk as the storage space, when the data block number of the first data block is blockid, in order to address the problem that a large number of blocks on a single disk can lead to slow access due to storing a large number of files in a single directory, the blockid can be rounded according to the number of directories in each alternative storage directory used to store each block, and the blockid can be converted into a hexadecimal string to obtain the data block storage location of the data block attribute information.

[0115] Based on the obtained data block storage locations, the first storage directory can be determined from among multiple alternative storage directories contained in the file subspace, and the blocks can be distributed and stored in the first storage directory.

[0116] For example, assuming the number of candidate storage directories for storing each block is 1000, the data conversion process is rounding, and the blockid is 100000, converting the blockid to a hexadecimal string gives 00000000000186A0. Rounding the blockid to the nearest integer gives 100000%1000=100. Therefore, the data block storage location is 100 / 00000000000186A0. Based on the data block storage location, the first storage directory can be obtained as / disk1 / block / 100 / 00000000000186A0.

[0117] Please refer to Figure 4A In the diagram of block hash storage, metadata in file format is stored in a list format. The data block attribute information corresponds to the block metadata. This data block attribute information may include the block's LBA address stored on disk, the block length, and may also include header information such as the block creation time and block fragment size, as well as CRC checksum information corresponding to the block data, etc., with no specific restrictions.

[0118] As one embodiment, the region attribute information includes the region offset position of the first data region, which is used to characterize the position of the first data region in the data subspace, for example, the region offset position is zoneoffset.

[0119] Similarly, continuing with the example of an SMR disk as the storage space, when the zone offset position of the first data area is zoneoffset, in order to address the problem of slow access caused by storing a large number of files in a single directory due to the large number of zones stored on a single disk, the zoneoffset can be rounded according to the number of directories used to store each alternative storage directory and the size of the zone, and the zoneoffset can be converted into a hexadecimal string to obtain the zone storage location of the zone attribute information.

[0120] Based on the obtained zone storage location, a first storage directory can be determined from multiple alternative storage directories contained in the file subspace, and the zones can be distributed and stored in that first storage directory.

[0121] For example, assuming the number of alternative storage directories for storing each zone is 1000, the data conversion process is rounded down, the zoneoffset is 32212254720, and the zone size is 256MB, converting the zoneoffset to a hexadecimal string gives 0000000780000000. Rounding the zoneoffset down gives (32212254720 / 256MB)%1000 = 120. Therefore, the data area storage location is 120 / 0000000780000000. Based on the data area storage location, the first storage directory can be obtained as / disk1 / zone / 120 / 0000000780000000.

[0122] Please refer to Figure 4B Zone metadata is stored in a list format. Zone attribute information corresponds to zone metadata information, which includes at least the zone's unique logical ID, the zone write length (consistent with the zone write pointer position), and a list of blocks contained within the zone. This may include the zone's creation time, etc., without specific restrictions. The list of blocks within the zone must include at least the block's offset address within the zone, its unique ID, and status information, stored sequentially. Other information can also be written, without specific restrictions.

[0123] As one embodiment, the file subspace contains a region pointer. When determining the first storage directory based on region attribute information and multiple alternative storage directories, an alternative storage directory can be determined according to the region storage location indicated by the region pointer in the file subspace, and this alternative storage directory can be used as the first storage directory. If the region storage location indicated by the region pointer is determined to be a location other than the multiple alternative storage directories, the first storage directory is created at the region storage location.

[0124] The zone pointer indicates the storage location of the zone attribute information to be stored. Continuing with the example of an SMR disk, when the first data area is a zone, and the last data block written within that zone is a block, the pointer following the block is the zone pointer. This pointer information within the zone indicates whether blocks can continue to be written within that zone. It is required that the zone write length remain consistent with the position of the zone write pointer.

[0125] Next, the first storage directory of the block is calculated based on the block ID. This calculation method is consistent with the method used to determine the first storage directory during storage. The specific method depends on the data conversion operation during storage. Then, the first storage directory of the appropriate zone is determined based on the zone pointer information for the block writing operation. The specific write operation is as follows:

[0126] When the storage location pointed to by the pointer is a candidate storage directory, this storage directory is used as the first storage directory for writing blocks. At this time, other blocks already exist in the zone. The blocks that need to be written are written sequentially, and the information of the written blocks is appended to the end of the zone metadata, that is, the position of the pointer is updated.

[0127] When the zone pointer indicates a storage location outside of the multiple alternative storage directories, a new storage directory is created within this indicated zone as the first storage directory, and the block is written to the newly created first storage directory. To ensure that metadata updates within the zone are consistent with block metadata updates, it is necessary to check whether the updates are consistent. If the update fails, a rollback operation is performed, and the write is re-attempted.

[0128] Please refer to Figure 5 There are two scenarios for performing a write operation:

[0129] S501: Calculate the first storage directory of the block based on the blockid. The calculation method is the same as that for hashing and storing the block metadata, so it will not be elaborated here. Then, determine the first storage directory of the zone based on the zone pointer information, and proceed with subsequent operations.

[0130] S502, determine whether the storage location pointed to by the pointer is a candidate storage directory. If the pointed location is a candidate storage directory, proceed to S503; if the pointed location is not a candidate directory, proceed to S505.

[0131] S503, the pointer indicates the candidate directory, which means that other blocks have been stored in the determined zone. The blocks to be stored can be written sequentially.

[0132] S504 requires updating the pointer position information after sequentially writing a block, appending the newly written block information to the end of the zone metadata.

[0133] S505: When the pointer indicates a location outside of multiple alternative storage directories, it means there is no suitable zone for block writing, and a new storage directory needs to be created as the first storage directory.

[0134] S506, write the block to the newly created storage directory.

[0135] S204, based on the data block attribute information and the region attribute information, write the first target data into the first data block, and store the first data block into the first data region.

[0136] Based on the indication of data block attribute information, the first target data is written into the first data block in the data subspace, and based on the indication of region attribute information, the first data block is stored into the first data region in the data subspace.

[0137] Taking block attribute information as block metadata and region attribute information as zone metadata as an example, based on block metadata and zone metadata, the first target data is allocated and written to the block. According to the length of the block, a zone is allocated to the block and the block is stored in the zone.

[0138] As one embodiment, after writing the first target data into the first data block based on the data block attribute information and the region attribute information, and storing the first data block in the first data region, the identification information of the second target data is obtained in response to the retrieval operation for the second target data.

[0139] Based on the identification information of the second target data, the second storage directories corresponding to the second data block and the second data area are determined in the file subspace, and the data block attribute information of the second data block and the data area attribute information of the second data area are read from the obtained second storage directories, wherein the second data block is used to store the second target data and the second data area is used to store the second data block.

[0140] Based on the data block attribute information of the second data block and the data region attribute information of the second data region, the second target data is read in the data subspace.

[0141] When the identification information includes the data block number of the second data block, which is used to uniquely identify the second data block, based on the identification information, the second storage directory corresponding to the second data block and the second data area is determined in the file subspace, and the data block attribute information of the second data block and the data area attribute information of the second data area are read from the obtained second storage directory, the data block number is converted according to the number of candidate storage directories contained in the file subspace to obtain the second storage directory of the second data block.

[0142] Read the data block attribute information of the second data block from the second storage directory of the second data block.

[0143] Based on the data block attribute information, determine the second storage directory of the second data area.

[0144] Read the region attribute information of the second data region from the second storage directory of the second data region.

[0145] Continuing with the example of an SMR disk as the storage space, when the second data block's block number is `blockid` and the second data region is a zone, in response to a retrieval operation targeting the second target data block, the retrieval can be performed based on the `blockid` in the target data's identifier. The corresponding second storage directory can be calculated from the `blockid`, allowing direct access to the corresponding block metadata information. This directory stores the block's LBA address across the entire disk. Since the zone has a fixed size, the zone offset (`zoneoffset`) can be calculated by dividing the block's LBA address across the entire disk by the known fixed size of the zone and then performing forensic processing. Based on the `zoneoffset`, it is converted into a string as the metadata filename using the aforementioned data conversion process. Further hashing can then be performed; the specific calculation and processing methods are not limited here. This calculates the zone metadata storage directory, thereby allowing access to the zone's metadata information.

[0146] For example, if the zone size is 256MB and the block's full-disk LBA address in the SMR disk is 1347420160000, then the zone offset position of the corresponding zone attribute information is calculated as zoneoffset = 1347420160000 / 256MB * 256MB = 1315700736, thus obtaining the corresponding zone metadata storage directory as / disk1 / zone / 000000004E6C0000.

[0147] As one embodiment, when the identification information includes the region offset position of the second data area, the region offset position represents the offset of the storage location of the region attribute information of the second data area relative to the region logical address, and the region logical address is the starting address of the data subspace, when the second storage directory corresponding to the second data block and the second data area is determined in the file subspace based on the identification information, and the data block attribute information of the second data block and the data area attribute information of the second data area are read from the obtained second storage directory, the region offset position is rounded according to the number of candidate storage directories contained in the file subspace to obtain the second storage directory of the second data area.

[0148] Read the region attribute information of the second data region from the second storage directory of the second data region;

[0149] Based on the region attribute information, determine the second storage directory of the second data block.

[0150] Read the data block attribute information of the second data block from the second storage directory of the second data block.

[0151] Continuing with the example of an SMR disk as the storage space, in response to a retrieval operation targeting the second target data block, on the other hand, a retrieval can be performed based on the zone offset of the second data area. The zone metadata's second storage directory is directly calculated from the zone offset. This operation is the same as the process described above and will not be repeated here. After obtaining the zone metadata through the same process as in the above embodiment, the block list information contained in the zone metadata information is obtained. The block list information contains the block ID, thus forming a closed loop. Similar to the retrieval process in the above embodiment, a retrieval is performed based on the block ID, and the corresponding second storage directory is calculated from the block ID, allowing direct access to the corresponding block metadata information.

[0152] The two aspects of searching described above can form a closed loop. Blocks and zones can be searched bidirectionally. This bidirectional searching method improves search efficiency and can be applied to operations such as deletion, making data processing more efficient. Please refer to [link / reference]. Figure 6 Bidirectional retrieval makes the process a closed loop.

[0153] As one embodiment, after performing the operation of writing the first target data into a block and storing the block into a zone, in response to the retrieval operation for the target data, the data retrieval includes retrieving block information and corresponding zone information based on blockid, retrieving zone metadata and block list information within the zone based on zoneoffset, and then retrieving the block information in reverse.

[0154] S601, in response to an operation on the second target data block, performs a retrieval operation on the second target data block based on the blockid.

[0155] S602, based on the obtained blockid, the corresponding second storage directory of block metadata can be calculated to access the block metadata. For details, please refer to the process of storing block metadata in the file subspace, which will not be repeated here.

[0156] S603, accessing the corresponding block metadata information, which contains relevant information about the block to be searched.

[0157] S604 is based on the block's LBA address on the entire disk, which is contained in the block's metadata information; that is, the address where the block is actually written to the disk.

[0158] S605 can perform a division and rounding operation between the block's LBA address and the known zone size of 256MB to calculate the corresponding zone offset.

[0159] S606: Based on the calculated zoneoffset, the second storage directory of the corresponding zone metadata can be obtained to access the zone metadata. For details, please refer to the steps when storing zone metadata in the file subspace, which will not be repeated here.

[0160] S607, access the corresponding zone metadata information, and retrieve the part related to the block from the zone metadata.

[0161] S608, the zone metadata information contains a block list, which includes the block ID. Obtaining the block ID returns S601. Thus, the retrieval process forms a closed loop.

[0162] As one embodiment, in response to a retrieval operation for a second target data block, when the retrieval is performed based on the zone offset of the second data area, starting from step S606, the second storage directory of the corresponding zone metadata can be obtained by calculation based on the zone offset, so as to access the zone metadata, execute step S607, access and obtain the corresponding zone metadata information, obtain the part related to the block in the zone metadata, and execute steps S601 to S604.

[0163] As one embodiment, after writing the first target data into the first data block and storing the first data block in the first data area based on the data block attribute information and the area attribute information, in response to the deletion operation of the third target data, the data block number of the third data block is obtained, and the third storage directory corresponding to the third data block and the third data area is determined in the file subspace, wherein the third data block is used to store the third target data, and the third data area is used to store the third data block.

[0164] Delete the data block attribute information of the third data block from the third storage directory corresponding to the third data block.

[0165] In the third storage directory corresponding to the third data area, delete the data block description information of the third data block contained in the region attribute information of the third data area.

[0166] Continuing with the example of an SMR disk, when the block number of the third data block is `blockid` and the third data area is a zone, to verify whether the block is within the zone where it was written, a search operation is first performed based on `blockid` to obtain the corresponding zone metadata. Based on the block offset information within the zone from the block list information contained in the zone metadata, the actual location of the block within the zone can be quickly retrieved using a binary search. The specific search method is not limited here.

[0167] If the block is not found, it means that the block is not in the third storage directory of the zone, and no deletion operation will be performed.

[0168] If the search is successful, it means that the block is in the third storage directory of the zone. In this case, the metadata file corresponding to the block needs to be deleted, the block status in the block list needs to be marked as deleted, and the number of blocks stored in the zone metadata needs to be decremented by 1.

[0169] After updating the number of blocks within the zone as described above, the pointer position within the zone needs to be adjusted to facilitate subsequent write operations. If the block count within the zone is not 0, no further operations are required; if the block count within the zone is 0, it indicates that the zone is empty, and the corresponding metadata file needs to be deleted, and the zone's pointer position needs to be reset.

[0170] Based on the number and status of blocks stored in a zone, it is possible to quickly determine whether a block within the zone is empty. If it is empty, a deletion operation is performed. Simultaneously, deletion operations can be performed quickly on blocks within the zone. This deletion operation, based on the storage scheme proposed in this application, makes the deletion operation more efficient.

[0171] Please refer to Figure 7 When performing a deletion operation, it is first necessary to perform a retrieval operation based on the block ID to obtain the block list information contained in the corresponding zone metadata.

[0172] S701, based on the block list in the corresponding zone metadata, obtains the offset information of the blocks stored therein within the zone.

[0173] S702, based on the corresponding block offset information within the zone, retrieves the actual location of the block within the zone using a binary search method. This actual location is the actual address obtained by adding the offset address to the initial address during storage.

[0174] S703, determine whether the block's location was found during the actual location retrieval. If the block's location was found, proceed to S704; otherwise, proceed to S705.

[0175] S704 After locating the block, proceed with the block deletion process. First, delete the metadata file corresponding to the block and mark the block status in the block list as deleted. Second, decrement the block count stored in the zone metadata by 1.

[0176] S705, the location of the block was not found, which means that the block is no longer stored in this zone, so no deletion operation will be performed.

[0177] S706: After decrementing the block count stored in the zone by 1, it is necessary to determine whether the block count in the zone is 0. If the count is 0, proceed to S707; if the count is not 0, proceed to S708.

[0178] S707, the block count in the zone is 0, which means the zone is empty. The metadata file corresponding to the zone needs to be deleted and the pointer position of the zone needs to be reset.

[0179] S708, if the count is not 0, then the operation ends.

[0180] Based on the same inventive concept, embodiments of this application provide a data processing apparatus capable of realizing the functions corresponding to the aforementioned data processing method. Please refer to... Figure 8 The device includes an acquisition module 801 and a processing module 802, wherein:

[0181] Acquisition module 801: Used to acquire the first target data in response to a write operation on the first target data;

[0182] Processing module 802: Based on the data length of the first target data, it allocates a first data block for the first target data in the data subspace of the storage space, and allocates a first data region for the first data block, and obtains the data block attribute information of the first data block and the region attribute information of the first data region respectively;

[0183] The processing module 802 is also used to: determine the corresponding first storage directory from multiple alternative storage directories contained in the file subspace of the storage space based on the data block attribute information and the region attribute information, and store the data block attribute information and the region attribute information into the corresponding first storage directory respectively;

[0184] The processing module 802 is also used to: write the first target data into the first data block based on the data block attribute information and the region attribute information, and store the first data block into the first data region.

[0185] In one possible embodiment, the data block attribute information includes a data block number of the first data block, the data block number being used to uniquely identify the first data block, and the processing module 802 is specifically used for:

[0186] Based on the number of directories in multiple alternative storage directories, the data block numbers are converted to obtain the data block storage location;

[0187] Based on the data block storage location, determine the first storage directory for the data block attribute information from multiple alternative storage directories;

[0188] Based on the region attribute information and multiple alternative storage directories, the first storage directory for the region attribute information is determined.

[0189] In one possible embodiment, the file subspace includes a region pointer, which indicates the region storage location for the region attribute information to be stored. The processing module 802 is specifically used for:

[0190] When the region storage location pointed to by the region pointer in the file subspace is determined to be a candidate storage directory, the candidate storage directory is used as the first storage directory;

[0191] When the region storage location indicated by the region pointer is determined to be a location other than multiple alternative storage directories, a first storage directory is created at the region storage location.

[0192] In one possible embodiment, the processing module 802 is specifically used for:

[0193] In response to a retrieval operation targeting the second target data, the identification information of the second target data is obtained;

[0194] Based on the identification information of the second target data, in the file subspace, the second storage directories corresponding to the second data block and the second data area are determined respectively, and the data block attribute information of the second data block and the data area attribute information of the second data area are read from the obtained second storage directories respectively. The second data block is used to store the second target data, and the second data area is used to store the second data block.

[0195] Based on the data block attribute information of the second data block and the data region attribute information of the second data region, the second target data is read in the data subspace.

[0196] In one possible embodiment, the processing module 802 is specifically used for:

[0197] Based on the number of alternative storage directories contained in the file subspace, the data block number is converted to obtain the second storage directory of the second data block;

[0198] Read the data block attribute information of the second data block from the second storage directory of the second data block;

[0199] Based on the data block attribute information, determine the second storage directory of the second data area;

[0200] Read the region attribute information of the second data region from the second storage directory of the second data region.

[0201] In one possible embodiment, the processing module 802 is specifically used for:

[0202] The second storage directory of the second data area is obtained by rounding down the region offset position according to the number of alternative storage directories contained in the file subspace.

[0203] Read the region attribute information of the second data region from the second storage directory of the second data region;

[0204] Based on the region attribute information, determine the second storage directory of the second data block;

[0205] Read the data block attribute information of the second data block from the second storage directory of the second data block.

[0206] In one possible embodiment, the processing module 802 is specifically used for:

[0207] In response to the deletion operation on the third target data, the data block number of the third data block is obtained. In the file subspace, the third storage directory corresponding to the third data block and the third data area is determined. The third data block is used to store the third target data, and the third data area is used to store the third data block.

[0208] Delete the data block attribute information of the third data block from the third storage directory corresponding to the third data block;

[0209] In the third storage directory corresponding to the third data area, delete the data block description information of the third data block contained in the region attribute information of the third data area.

[0210] Please refer to Figure 9The aforementioned data processing apparatus can run on a computer device 900. The current and historical versions of the data storage program, as well as the application software corresponding to the data storage program, can be installed on the computer device 900, which includes a processor 980 and a memory 920. In some embodiments, the computer device 900 may include a display unit 940, which includes a display panel 941 for displaying a user-interactive interface, etc.

[0211] In one possible embodiment, the display panel 941 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0212] The processor 980 is used to read a computer program and then execute the methods defined by the computer program. For example, the processor 980 reads a data storage program or file, thereby running the data storage program on the computer device 900 and displaying the corresponding interface on the display unit 940. The processor 980 may include one or more general-purpose processors, and may also include one or more DSPs (Digital Signal Processors) for performing related operations to implement the technical solutions provided in the embodiments of this application.

[0213] The memory 920 generally includes main memory and secondary storage. Main memory can be random access memory (RAM), read-only memory (ROM), and cache, etc. Secondary storage can be a hard disk, optical disk, USB flash drive, floppy disk, or magnetic tape drive, etc. The memory 920 is used to store computer programs and other data. The computer programs include applications corresponding to each client, and other data may include data generated after the operating system or applications are run, including system data (e.g., operating system configuration parameters) and user data. In this embodiment, program instructions are stored in the memory 920, and the processor 980 executes the program instructions in the memory 920 to implement any of the methods described in the preceding figures.

[0214] The aforementioned display unit 940 is used to receive input digital information, character information, or contact touch operations / non-contact gestures, and to generate signal inputs related to user settings and function control of the computer device 900. Specifically, in this embodiment, the display unit 940 may include a display panel 941. The display panel 941, for example, is a touch screen, which can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or on the display panel 941), and drive corresponding connection devices according to a pre-set program.

[0215] In one possible embodiment, the display panel 941 may include two parts: a touch detection device and a touch controller. The touch detection device detects the player's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 980. It can also receive and execute commands from the processor 980.

[0216] The display panel 941 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 940, in some embodiments, the computer device 900 may also include an input unit 930. The input unit 930 may include an image input device 931 and other input devices 932, wherein the other input devices may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0217] In addition to the above, the computer device 900 may also include a power supply 990 for powering other modules, an audio circuit 960, a near-field communication module 970, and an RF circuit 910. The computer device 900 may also include one or more sensors 950, such as an accelerometer, a light sensor, and a pressure sensor. The audio circuit 960 specifically includes a speaker 961 and a microphone 962, for example, the computer device 900 can use the microphone 962 to collect the user's voice and perform corresponding operations.

[0218] As one embodiment, the number of processors 980 can be one or more, and the processors 980 and the memory 920 can be coupled together or relatively independent.

[0219] As one example, Figure 9 The processor 980 in the middle can be used to implement, for example Figure 8 The functions of the acquisition module 801 and the processing module 802 in the process.

[0220] As one example, Figure 9 The processor 980 in the text can be used to implement the functions of the server or terminal devices discussed above.

[0221] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0222] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of software products, for example, through a computer program product. This computer program product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0223] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0224] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data processing method, characterized in that, Applied to distributed object storage systems, including: In response to a write operation on the first target data, the first target data is obtained; Based on the data length of the first target data, the first target data is written into a first data block. Based on the size of the first data block, a first data region is allocated to the first data block from multiple data regions contained in the data subspace. Data block attribute information of the first data block and region attribute information of the first data region are obtained respectively. The data block attribute information includes the data block number of the first data block. The region attribute information includes the region offset position of the first data region, and the region offset position represents the position of the first data region in the data subspace. Based on the number of directories in each of the multiple alternative storage directories contained in the file subspace of the storage space, which are used to store data block attribute information, the data block number is processed by data conversion to obtain the data block storage location; the file subspace has been formatted as a standard file system; Based on the number of directories in each of the multiple alternative storage directories used to store regional attribute information, the regional offset position is processed by data conversion to obtain the regional storage location; Based on the storage location of the data block, determine the first storage directory corresponding to the data block attribute information, and based on the storage location of the region, determine the first storage directory corresponding to the region attribute information, and store the data block attribute information and the region attribute information in the corresponding first storage directories respectively. Based on the data block attribute information and the region attribute information, the first data block is stored in the first data region.

2. The method according to claim 1, characterized in that, The file subspace contains a region pointer, which is used to indicate the region storage location of the region attribute information to be stored. The step of determining the first storage directory for the regional attribute information based on the regional attribute information and the plurality of alternative storage directories includes: When the region storage location indicated by the region pointer in the file subspace is determined to be a candidate storage directory, the candidate storage directory is used as the first storage directory; When it is determined that the region storage location indicated by the region pointer is a location other than the plurality of alternative storage directories, the first storage directory is created at the region storage location.

3. The method according to claim 1 or 2, characterized in that, After writing the first target data into the first data block and storing the first data block in the first data region based on the data block attribute information and the region attribute information, the method further includes: In response to a retrieval operation targeting the second target data, the identification information of the second target data is obtained; Based on the identification information of the second target data, in the file subspace, the second storage directories corresponding to the second data block and the second data area are determined respectively, and the data block attribute information of the second data block and the data area attribute information of the second data area are read from the obtained second storage directories respectively, wherein the second data block is used to store the second target data, and the second data area is used to store the second data block; Based on the data block attribute information of the second data block and the data region attribute information of the second data region, the second target data is read in the data subspace.

4. The method according to claim 3, characterized in that, The identification information includes the data block number of the second data block, which is used to uniquely identify the second data block; Based on the identification information, the step of determining the second storage directory corresponding to the second data block and the second data region in the file subspace, and reading the data block attribute information of the second data block and the data region attribute information of the second data region from the obtained second storage directory, includes: Based on the number of candidate storage directories contained in the file subspace, the data block number is converted to obtain the second storage directory of the second data block; Read the data block attribute information of the second data block from the second storage directory of the second data block; Based on the data block attribute information, determine the second storage directory of the second data area; Read the region attribute information of the second data region from the second storage directory of the second data region.

5. The method according to claim 3, characterized in that, The identification information includes the region offset position of the second data area, which represents the offset of the storage location of the region attribute information of the second data area relative to the region logical address, where the region logical address is the starting address of the data subspace. Based on the identification information, the step of determining the second storage directory corresponding to the second data block and the second data region in the file subspace, and reading the data block attribute information of the second data block and the data region attribute information of the second data region from the obtained second storage directory, includes: The offset position of the region is rounded up according to the number of candidate storage directories contained in the file subspace to obtain the second storage directory of the second data region; Read the region attribute information of the second data region from the second storage directory of the second data region; Based on the region attribute information, determine the second storage directory of the second data block; Read the data block attribute information of the second data block from the second storage directory of the second data block.

6. The method according to claim 1 or 2, characterized in that, The region attribute information includes the data block description information of each data block stored in the corresponding data region within the data subspace; After writing the first target data into the first data block and storing the first data block in the first data region based on the data block attribute information and the region attribute information, the method further includes: In response to a deletion operation on the third target data, the data block number of the third data block is obtained, and in the file subspace, the third storage directory corresponding to the third data block and the third data area is determined, wherein the third data block is used to store the third target data, and the third data area is used to store the third data block; Delete the data block attribute information of the third data block from the third storage directory corresponding to the third data block; In the third storage directory corresponding to the third data region, delete the data block description information of the third data block contained in the region attribute information of the third data region.

7. A data processing apparatus, characterized in that, Applied to distributed object storage systems, including: Acquisition module: used to acquire the first target data in response to a write operation on the first target data; Processing module: Used to write the first target data into a first data block based on the data length of the first target data, and to allocate a first data region to the first data block from multiple data regions contained in the data subspace based on the size of the first data block, thereby obtaining data block attribute information of the first data block and region attribute information of the first data region; the data block attribute information includes the data block number of the first data block; the region attribute information includes the region offset position of the first data region, the region offset position representing the position of the first data region in the data subspace; The processing module is further configured to: perform data conversion processing on the data block number according to the number of directories in each of the multiple alternative storage directories used to store data block attribute information in the file subspace of the storage space, to obtain the data block storage location; the file subspace has been formatted as a standard file system; and perform data conversion processing on the region offset position according to the number of directories in each of the multiple alternative storage directories used to store region attribute information, to obtain the region storage location; The processing module is further configured to: determine the first storage directory corresponding to the data block attribute information based on the data block storage location, and determine the first storage directory corresponding to the region attribute information based on the region storage location, and store the data block attribute information and the region attribute information into the corresponding first storage directories respectively; The processing module is further configured to: store the first data block into the first data region based on the data block attribute information and the region attribute information.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

9. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 6 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Block-chain data storage method, device, equipment and medium

    CN109086388A

  • Space distribution method of shingled hard disk, file storage system and server

    CN113157600A