Data processing method and apparatus, storage medium, and electronic device

Abstract

Embodiments of the present application provide a data processing method and device, a storage medium and an electronic device. The method comprises: in response to a data write operation, obtaining unit description information of a storage unit allocated for a data object to be written; in a case where the unit description information indicates that a reference data amount that can be stored by a remaining storage space of the storage unit is greater than a first data amount of the data object, creating a bitmap object matched with the storage unit, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit; and in a case where the data object is written into at least one target storage subspace in the plurality of storage subspaces, performing a set operation on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate the storage state of the at least one target storage subspace.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a data processing method and apparatus, a storage medium, and an electronic device. Background Art

[0002] In cloud computing and virtualization scenarios, the file system is a core technology for building storage pools. File systems divide backend data disks into data blocks (metadata) of a certain size to manage space and organize data. The ever-increasing storage pools in cloud computing scenarios and the increasing number of large virtual disk applications pose significant challenges to file system metadata management.

[0003] However, in existing file system technology, the backend storage space allocated by the file system to the virtual machine does not initialize data at the time of allocation. Therefore, the backend storage space of the virtual machine will contain a large amount of invalid data. When the virtual machine writes data to a data block for the first time, in order to avoid subsequently reading invalid data in the data block, the file system will perform a zero operation on the part of the data block that does not store the written data. That is to say, for the memory, in addition to writing the target data that needs to be written, several zero values ​​are also written. This process is also called write amplification. That is to say, in the related technology, there is a problem of write amplification in the process of writing data.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a data processing method and apparatus, a storage medium, and an electronic device to at least solve the technical problem of write amplification caused by data writing existing in the related art.

[0006] According to one embodiment of the present application, a data processing method is provided, comprising: in response to a data write operation, obtaining unit description information of a storage unit allocated for a data object to be written;

[0007] When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created, wherein multiple bits in the bitmap object are respectively used to indicate the storage status of multiple storage subspaces in the storage unit; when the data object is written to at least one target storage subspace among the multiple storage subspaces, a set operation is performed on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate the storage status of at least one target storage subspace.

[0008] According to another aspect of an embodiment of the present application, there is further provided a data processing apparatus, comprising: an information acquisition unit, configured to acquire, in response to a data write operation, unit description information of a storage unit allocated to a data object to be written;

[0009] a bitmap creation unit, configured to create a bitmap object that does not match the storage unit if the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit;

[0010] A setting unit is used to perform a setting operation on at least one target bit in a bitmap object when writing a data object to at least one target storage subspace in a plurality of storage subspaces, wherein the at least one target bit is used to indicate the storage status of the at least one target storage subspace.

[0011] Optionally, the above-mentioned data processing device also includes: a field update unit, used to obtain a first storage location of a data structure for storing a bitmap object; updating the data storage location field in the unit description information according to the first storage location, wherein the data storage location field before the update is used to indicate the second storage location of the storage unit; updating the target identification field in the unit description information, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the target identification field after the update is used to indicate that the data storage location field points to the data structure.

[0012] Optionally, the above-mentioned data processing device also includes: an information determination unit, used to: obtain the second storage position of the storage unit, and determine the second storage position as the first descriptive information in the data structure; determine the number of digits in the bitmap object as the second descriptive information in the data structure; determine the bitmap object as the third descriptive information in the data structure.

[0013] Optionally, the above-mentioned data processing device also includes: a writing unit, which is used to: write the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object; write the first data part of the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, and obtain the unit description information of the reference storage unit allocated to the second data part of the data object.

[0014] Optionally, the above-mentioned bitmap creation unit includes one of the following: a first determination module, used to: determine the target quantity as the number of bits included in the bitmap object, wherein the target quantity is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data of the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data; a second determination module, used to obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

[0015] Optionally, the above-mentioned data processing device also includes: a data reading unit, used to: in response to a data reading operation, obtain unit description information of at least one storage unit for storing a data object; in a case where the target identification field in the unit description information indicates that the data storage location field points to a data structure, determine the data structure according to the storage location field; obtain a second storage location for indicating the storage unit from the data structure, and obtain a reference bitmap object from the data structure; in a case where at least one reference bit in the reference bitmap object is set, obtain at least one data object from the storage unit located at the second storage position according to the storage subspace corresponding to at least one bit.

[0016] Optionally, the above-mentioned data processing device also includes: a bitmap writing unit, which is used to: when the target identification field in the unit description information indicates that the data storage location field points to a data structure, allocate at least one reference storage subspace for the data object to be written from multiple storage subspaces, wherein the bit corresponding to the reference storage subspace is not set; when writing the data object to at least one reference storage subspace, perform a set operation on the bit corresponding to the non-reference storage subspace in the bitmap object.

[0017] Optionally, the above-mentioned data processing device also includes: a bitmap clearing unit, which is used to: when the current system load parameters meet the target conditions, determine the current storage unit to be processed from multiple storage units configured with their own matching bitmap objects; clear the storage space in the current storage unit that does not store valid data; clear the bitmap objects that do not match the current storage unit; and update the current unit description information of the current storage unit, wherein the updated current unit description information is used to indicate the storage status of the current storage unit.

[0018] According to another aspect of the embodiments of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above data processing method.

[0019] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the data processing method through the computer program.

[0020] Through the present application, first, in response to a data write operation, unit description information of a storage unit allocated to a data object to be written is obtained. When the unit description information indicates that the reference data amount that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created. Each bit in the bitmap object corresponds to a storage subspace in the storage unit, thereby determining the storage status of multiple storage subspaces in the storage unit by the setting status of multiple bits in the bitmap object. Specifically, when a data object is written to a storage subspace in the storage unit, the corresponding bit of the storage subspace will be set, while the bit of the storage subspace where the data is not written will not be set. Then, during the data reading process, the file system can return a value of 0 based on the setting status of the target bit if the target bit is not set, without having to set the portion of the storage unit that does not store the data to be written to zero to avoid reading invalid data. In this solution, when the amount of data to be written is smaller than the amount of data in the storage unit, a bitmap object that does not match the storage unit is created. Only the data to be written needs to be written to the storage unit, and there is no need to additionally write a 0 value to the storage unit to avoid reading invalid data. No data other than the data to be written will be written. In other words, the technical problem of write amplification accompanying data writing in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0022] Figure 1 This is a hardware structure block diagram of a server device for a data processing method according to an embodiment of the present application;

[0023] Figure 2 is a flow chart of a data processing method according to an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of a data processing method according to an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of another data processing method according to an embodiment of the present application;

[0026] Figure 5 is a structural diagram of a data processing device according to an embodiment of the present application;

[0027] Figure 6 is a structural diagram of a data processing electronic device according to an embodiment of the present application;

[0028] Figure 7 is a flowchart of another data processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0031] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device of a data processing method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 The server device includes a processor 102 (only one of which is shown) (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. The server device may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components than shown, or with different Figure 1 Different configurations shown.

[0032] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a method for adjusting a read voltage in a memory in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory arranged relative to the processor 102 process, and these process memories may be connected to a server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] Transmission device 106 is used to receive or transmit data via a network. A specific example of such a network may include a wireless network provided by a communication provider of a server device. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] As an optional implementation, Figure 2 As shown, the above data processing method includes:

[0035] S202, in response to a data write operation, obtaining unit description information of a storage unit allocated for a data object to be written;

[0036] S204: if the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, create a bitmap object that does not match the storage unit, wherein multiple bits in the bitmap object are respectively used to indicate storage states of multiple storage subspaces in the storage unit;

[0037] S206 , when writing the data object into at least one target storage subspace among the multiple storage subspaces, performing a set operation on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate the storage status of the at least one target storage subspace.

[0038] It should be noted that the above-mentioned storage unit is the smallest unit for storing file data in the file system. When writing data, the above-mentioned step S202 is triggered: in response to the data writing operation, the unit description information of the storage unit allocated for the data object to be written is obtained. It should be noted that the above-mentioned storage unit allocated for the data object to be written is the storage unit allocated by the file system for storing the data to be written when there is data to be written. In addition, the above-mentioned description information includes the amount of data that can be stored in the remaining storage space of the above-mentioned storage unit;

[0039] Then, step S204 is executed. If the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created, wherein multiple bits in the bitmap object are respectively used to indicate the storage status of multiple storage subspaces in the storage unit;

[0040] It should be noted that the above bitmap object includes multiple bits, each bit corresponds to a storage subspace in the storage unit, and the above operation of creating a bitmap object that does not match the storage unit includes one of the following two methods:

[0041] Method 1: Determine the target number as the number of bits included in the bitmap object, where the target number is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data in the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data;

[0042] Method 2: Obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

[0043] In the above-mentioned method 1, the ratio of the total amount of data that can be stored in the storage unit to the minimum amount of modified data in the current file management system is used as the number of bits of the above-mentioned bitmap object. In the above-mentioned method 2, the ratio of the total amount of data that can be stored in the storage unit to the first amount of data of the data object to be stored is used as the number of bits of the above-mentioned bitmap object.

[0044] In an optional embodiment, when the first data volume of the above-mentioned data object is less than a preset threshold, the ratio between the total data volume that can be stored in the storage unit and the minimum modified data volume of the current file management system is obtained, and the ratio is used as the number of bits of the above-mentioned bitmap object.

[0045] In another optional embodiment, when the first data amount of the above-mentioned data object is greater than a preset threshold, the ratio between the total data amount that can be stored in the storage unit and the first data amount of the data object is obtained, and the ratio between the total data amount and the first data amount is further determined as the number of bits included in the bitmap object.

[0046] Through the above two methods, a data bitmap containing multiple bits can be determined, and each bit can correspond to a storage space in the storage unit. It can be understood that, for the above method one, the size of the storage subspace corresponding to each bit is the minimum data modification amount of the current file management system, and for the above method two, the size of the storage subspace corresponding to each bit is the size of the data to be written, thereby realizing the dynamic determination of the number of bits of the bitmap, so that in the process of bitmap creation, a scheme with a small number of bit generation can be dynamically selected according to the size of the data to be written, thereby reducing the overhead of the file system and improving data processing efficiency.

[0047] Then, the above step S208 can be executed. When the data object is written to at least one target storage subspace in multiple storage subspaces, a set operation is performed on at least one target bit in the bitmap object, wherein the at least one target bit is used to indicate the storage status of the at least one target storage subspace.

[0048] It should be noted that the above-mentioned set operation is to set the value of the target bit to 1. It can be understood that the value of the above-mentioned target bit before the set operation is performed is 1. Each target bit is used to indicate the storage status of the corresponding target storage subspace. When the value of the target bit is 1, it indicates that the corresponding target subspace stores write data. When the value of the target bit is 0, it indicates that the corresponding target subspace does not store write data. It should be noted that the target subspace does not store write data is not equivalent to no data being stored. In fact, the target subspace may store invalid data. In the case of storing invalid data, the corresponding target bit is not set and the value is 0. After the data to be written is written into the target subspace, the target subspace stores the write data. In this case, the target bit is set and the value is 1.

[0049] Through the above-mentioned method of the present application, first, in response to a data write operation, unit description information of the storage unit allocated to the data object to be written is obtained. When the unit description information indicates that the reference data amount that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created, wherein multiple bits in the bitmap object are respectively used to indicate the storage status of multiple storage subspaces in the storage unit; when the data object is written to at least one target storage subspace among the multiple storage subspaces, a set operation is performed on at least one target bit in the bitmap object, that is, the target bit of the storage subspace where the data is not written is not set, thereby enabling the file system to return a value of 0 based on the setting status of the target bit when the target bit is not set, without having to avoid reading dirty data by setting the portion of the storage unit that does not store the data to be written to zero. This solution, which only requires writing the data to be written to the memory without additionally writing a value of 0, solves the technical problem of write amplification when writing data in the prior art.

[0050] As an optional implementation, after creating a bitmap object that does not store unit matching, the method further includes:

[0051] S1, obtaining a first storage location of a data structure for storing a bitmap object;

[0052] S2, updating the data storage location field in the unit description information according to the first storage location, wherein the data storage location field before the update is used to indicate the second storage location of the storage unit;

[0053] S3. Update the target identification field in the unit description information, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the target identification field after the update is used to indicate that the data storage location field points to the data structure.

[0054] It should be noted that, in the above step S1, the data structure of the bitmap object refers to a custom data type including bitmap object fields. For example, in an optional embodiment, there is the following data structure:

[0055]

[0056] The above ocfs2_indi_bm_rec is a data structure, which is a user-defined C++ data type, and ir_bm is a bitmap object variable.

[0057] In an optional embodiment, after executing the above step S1 and obtaining the storage location of the data structure, execute the above step S2 to update the data storage location field of the above unit description information, so that the data location storage field is changed from indicating the storage location of the storage unit to indicating the storage location of the data structure.

[0058] Then execute step S3 to update the target representation field in the unit description information. It should be noted that the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and after the above step S2 is executed, the above data storage location field is changed from indicating the storage location of the storage unit to indicating the storage location of the data structure, that is, after the above step S2 is executed, the above data storage location field is changed from pointing to the storage unit to pointing to the data structure, so it is necessary to update the target representation field pointed to by the data storage location field so that the updated target representation field indicates that the above data storage location field points to the data structure.

[0059] The following combination Figure 3 、 Figure 4 The above steps S1-S3 are explained.

[0060] Figure 3 This is a schematic diagram of the underlying file data organization of a file system. Figure 3 In the file system, a file data mainly includes identification information 302, which is used to record key information of the file, such as the file size, the file creation time, etc. A file data also includes a data node tree composed of multiple data nodes, and multiple storage units. The storage unit is the smallest unit for recording user data. In the file system, a file will be composed of multiple discrete storage units of different lengths.

[0061] Figure 3 In the example, the data storage location field 324 points to the storage unit 352, and the data storage location field 326 points to the storage unit 354. After the bitmap object is created for the storage unit 354, the data storage location field 326 changes from indicating the storage location of the storage unit to indicating the storage location of the structure of the bitmap object. Figure 4 As shown, data storage location field 326 no longer directly points to storage unit 354, but instead points to bitmap object 412 corresponding to storage unit 354. Each bit of bitmap object 412 points to a corresponding storage subspace. Furthermore, the file system also includes a target identification field (not shown). This target identification field is initially used to indicate that data storage location field 326 points to storage unit 354. After bitmap object 412 is created, it is updated. The updated target identification field indicates that data storage location field 326 points to bitmap object 412.

[0062] Through the above-mentioned implementation of the present application, the first storage location of the data structure for storing the bitmap object is first obtained, and then the data storage location field in the unit description information is updated according to the first storage location, and the target identification field in the unit description information is updated so that the updated target identification field indicates that the data storage location field points to the data structure.

[0063] As an optional implementation, before obtaining the storage location of the data structure for storing the bitmap object, the method further includes:

[0064] S1, obtaining a second storage location of the storage unit, and determining the second storage location as first description information in the data structure;

[0065] S2, determining the number of digits in the bitmap object as second description information in the data structure;

[0066] S3: Determine the bitmap object as the third description information in the data structure.

[0067] It should be noted that the first description information, the second description information, and the third description information determined in the above steps S1 to S3 are three variables of the above data structure. For example, the above data structure can be in the following form:

[0068]

[0069] Among them, ir_blkno is the first descriptive information, pointing to the actual address of the storage unit (the second storage location), ir_count is the second descriptive information, indicating the number of bits of the bitmap object, and ir_bm is the third descriptive information, indicating the data bitmap. It can be understood that a data bitmap is an array, and there are multiple objects in an array. It should be noted that each object is 8 bits, and the bit of each array object corresponds to a minimum write unit.

[0070] As an optional implementation manner, after obtaining the unit description information of the storage unit allocated for the data object to be written, the method further includes:

[0071] S1, writing the data object into the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object;

[0072] S2. When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, the first data part of the data object is written in the remaining storage space of the storage unit, and the unit description information of the reference storage unit allocated to the second data part of the data object is obtained.

[0073] It can be understood that after obtaining the description information of the storage unit allocated to the data object to be written, the data object to be written will choose to write one or more storage units based on its own data volume. When the amount of data that can be stored in the above-mentioned storage unit is equal to the amount of data of the data object to be written, the data object to be written will be written into the remaining writable storage space in the above-mentioned storage unit that does not store other write data. When the amount of data of the data object to be written itself is greater than the amount of data that can be stored in the above-mentioned storage unit, the above-mentioned data object to be written will store part of its own data in the storage unit according to the amount of data that can be stored in the storage unit, and for the unstored part, a new storage unit will be allocated, and the description information of the new storage unit will be obtained. The unstored part will be stored according to the storable data volume indicated by the description information of the new storage unit. If there is still an unstored part, the iteration will continue until all the data to be written is stored or there is no storage unit in the file system that can store new data. The following is an example to explain.

[0074] For example, in an optional embodiment, it is assumed that the data object A to be written currently has 5 units of length of data, and the file system allocates storage unit A for the data to be written currently, and storage unit A has two units of length of data storage space. In this case, the two units of length of data object A to be written are stored in storage unit A as the first data part, and the remaining three units of length of data are used as a new data object B to be written (the second data part). A storage unit B is further allocated to the new data object B to be written, and storage unit B has two units of length of data storage space. In this case, the two units of length of data object B are stored in storage unit B as the first data part, and the remaining one unit of length of data is used as a new data object C to be written. A storage unit C is further allocated to data object C, and storage unit C has two units of length of data storage space. Unit length of storable data space. In this case, data object C is completely stored in storage unit C, and the iteration stops at this time. At the same time, the bitmap object creation operation of storage unit C is triggered to create a bitmap object C corresponding to storage unit C. It is assumed that the number of bits of bitmap object C at this time is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data in the current file management system. When the minimum amount of modified data is half of the above unit length, it is determined that bitmap object C has four bits, and each bit corresponds to a storage subspace in storage unit C. It can be understood that the above data object C is stored in a storage subspace in storage unit C, and the bit corresponding to the storage subspace will be set to indicate that valid data is stored in the storage subspace, while the other unset bits indicate that no data is stored in the corresponding storage subspace or invalid data is stored.

[0075] Through the above-mentioned implementation mode of the present application, when the amount of data to be written currently exceeds the amount of data that can be stored in the allocated storage unit, the data to be written can be stored in multiple storage units through a loop iteration method. While the data object to be written currently is completely stored, during the last iteration, when the amount of data in the allocated storage unit is greater than the amount of data to be written in this iteration, the bitmap creation is triggered, thereby reducing the operating overhead of the file system and improving the operating efficiency.

[0076] As an optional implementation manner, when writing the data object into at least one target residual subspace among the plurality of residual subspaces, after performing a setting operation on at least one target bit in the bitmap object, the method further includes:

[0077] S1, in response to a data read operation, obtaining unit description information of at least one storage unit for storing a data object;

[0078] S2-1, when the target identification field in the unit description information indicates that the data storage location field points to the storage unit, and the unit description information indicates that the storage unit stores the first reference data object that has been written, determining the first reference data object in the storage unit as the data object;

[0079] S2-2, when the target identification field in the unit description information indicates that the data storage location field points to the storage unit, and the unit description information indicates that the first reference data object to be written is not stored in the storage unit, determining a zero value as the data object;

[0080] S2-3, when the target identification field in the unit description information indicates that the data storage location field points to a data structure, determining the data structure according to the storage location field;

[0081] S3, obtaining a second storage location indicating the storage unit from the data structure, and obtaining a reference bitmap object from the data structure;

[0082] S4: When at least one reference bit in the reference bitmap object is set, obtain at least one data object from the storage unit located at the second storage position according to the storage subspace corresponding to at least one of the bits.

[0083] It should be noted that the bitmap object is created when the data amount of the data object to be written is less than the data amount that can be stored in the storage unit. That is to say, when the data amount of the data object to be written is equal to the data amount that can be stored in the storage unit, the bitmap object will not be created. Therefore, in the above step S1, the storage unit of the above data object to be read may have a corresponding bitmap object or may not have a corresponding bitmap object.

[0084] It should be further explained that when part of the storage space of the storage unit stores user-written data, there is a corresponding bitmap object in the storage unit. At this time, the data to be read is determined based on the setting status of each bit of the bitmap object. When the bit is set, it means that the storage subspace corresponding to the bit stores the data written by the user. At this time, the data in the storage subspace is determined as the data to be read. When the bit of the bitmap object is not set, it means that the storage subspace corresponding to the bit stores user-written data. At this time, invalid data may be stored in the storage subspace. Therefore, the data in the storage subspace cannot be used as the data to be read, but the zero value is used as the data to be read.

[0085] In addition, when a storage unit does not have a corresponding bitmap object, it means that all storage spaces of the storage unit have stored the data written by the user, or all storage spaces of the storage unit have not stored the data written by the user. When all storage spaces have stored the data written by the user, the storage data in the storage unit is used as the data to be read. When all storage spaces of the storage unit have not stored the data written by the user, invalid data may be stored in the above storage unit. Therefore, the data in the storage subspace cannot be used as the data to be read at this time, but the zero value is used as the data to be read.

[0086] The following is an example explanation of steps S1 to S4:

[0087] First, obtain the unit description information of at least one storage unit that stores the data object to be read. Assuming that data needs to be read from three storage units A, B, and C at this time, obtain the unit description information corresponding to each of the storage units A, B, and C. Assuming that the unit description information of storage unit A indicates that storage unit A has no corresponding bitmap object, and the data in storage unit A is data written by the user, it means that all the data in storage unit A is data written by the user. At this time, all the data in storage unit A (the first data object) is used as the data object.

[0088] Assume that the unit description information of storage unit B indicates that storage unit A has no corresponding bitmap object, and the data in storage unit B is not data written by the user, and in this case, a zero value is used as the data object.

[0089] Assume that the target identification field in the unit description information of storage unit C indicates that the data storage location field points to a data structure. In this case, the data storage location field no longer points to storage unit C, but instead points to the data structure. Furthermore, a second storage location indicating storage unit C is obtained from the data structure, the data structure is instantiated, and the bitmap variable in the instantiated data structure is called to generate storage unit C. At this point, the target reference data object is determined based on the setting status of each target bit of bitmap object C. Specifically, assuming that bitmap object C has two target bits: target bit A and target bit B, and assuming that target bit A is set and target bit B is not set, the data stored in the storage subspace corresponding to target bit A is used as the data object corresponding to target bit A, and a value of zero is used as the data object corresponding to target bit B. Thus, a read operation on the aforementioned data object is implemented.

[0090] Through the above-mentioned implementation mode of the present application, data can be read according to the data storage situation of the storage unit and the corresponding bit position situation. When the storage unit completely stores the data written by the user, the data in the storage unit is returned. When the storage unit does not store the data written by the user, a value of 0 is returned. When part of the storage subspace in the storage unit stores the data written by the user, data is returned according to the setting situation of each bit of the bitmap of the storage unit. If the bit corresponding to the storage subspace where the data to be read is located is set, the data in the storage subspace is returned. When the bit corresponding to the storage subspace where the data to be read is located is not set, a value of 0 is returned. This makes it unnecessary to avoid reading dirty data by setting the part of the storage unit that does not store the user-written data to zero, and solves the technical problem of write amplification when writing data in the prior art.

[0091] As an optional implementation manner, in response to the data writing operation, after obtaining the unit description information of the storage unit allocated to the data object to be written, the method further includes:

[0092] S1, when the target identification field in the unit description information indicates that the data storage location field points to a data structure, allocating at least one reference storage subspace for the data object to be written from the plurality of storage subspaces, wherein the target bit corresponding to the reference storage subspace is not set;

[0093] S2: When writing the data object into at least one reference storage subspace, performing a setting operation on a target bit corresponding to the non-reference storage subspace in the bitmap object.

[0094] It can be understood that in the above step S1, when the storage unit allocated to the storage unit to be stored is an allocated bitmap, the target identification field in the unit description information of the above storage unit indicates that the data storage location field points to the data structure. At this time, according to the bit position of the bitmap object corresponding to the storage unit, a storage subspace is further allocated for the data object to be written.

[0095] It can be understood that the set bit in the bitmap object represents that the storage subspace corresponding to the bit has stored valid data, while the unset bit represents that the corresponding storage subspace does not store valid data. At this time, the storage subspace corresponding to the unset bit can be allocated to the data object to be written to store the data object to be written. When the data object to be stored is written into the allocated storage subspace, the bit corresponding to the storage subspace of the written data object is set.

[0096] Through the above-mentioned implementation of the present application, when the allocated storage unit is a storage unit for which a bitmap has been created, the storage subspace corresponding to the unset bit is allocated to the data object to be written, and the allocated storage subspace is set. Through the above-mentioned implementation of the present application, the data writing operation on the storage unit for which a bitmap has been created can be realized.

[0097] As an optional implementation, after performing a setting operation on at least one target bit in the bitmap object, the method further includes:

[0098] S1, when the current system load parameter meets the target condition, determining a current storage unit to be processed from a plurality of storage units configured with respective matching bitmap objects;

[0099] S2, clearing the storage space in the current storage unit that does not store valid data;

[0100] S3, clear the bitmap objects that do not match the current storage unit;

[0101] S4. Update the current unit description information of the current storage unit, wherein the updated current unit description information is used to indicate the storage status of the current storage unit.

[0102] It should be noted that the bitmap object itself is an index of the file system. The more bitmap objects are created, the greater the index overhead of the file system. Therefore, for the storage unit of the created bitmap object, when the parameters of the current system meet the target conditions, the storage subspace corresponding to the unset bits in the storage unit, that is, the storage space in the above step S2 that does not store valid data, will be cleared. Specifically, the operation of writing the value of 0 to the above storage space that does not store valid data is performed.

[0103] When the writing of the value 0 is completed, there is no invalid data in the storage unit, and therefore, no invalid data will be read from the storage unit. Therefore, there is no need to determine the read data by whether the bit in the bitmap is set. At this time, the above step S3 can be executed to clear the bitmap object corresponding to the current storage space, and then step S4 can be executed to update the current unit description information of the current storage unit, so that the updated current unit description information indicates that no invalid data is stored in the current storage unit.

[0104] Through the above-mentioned implementation of the present application, when the current system load parameters meet the target conditions, the current storage unit to be processed is determined from multiple storage units configured with their own matching bitmap objects. Furthermore, the storage space in the current storage unit that does not store valid data is cleared to ensure that there is no invalid data in the current storage unit, and the bitmap object can be cleared to reduce the index overhead of the current system.

[0105] The following combination Figure 3 、 Figure 4 、 Figure 7 A complete data processing method is described.

[0106] Assume that the current file system is OCFS2 file system, Figure 3 The storage unit 354 in the file is a storage unit that has been allocated backend storage space but has not yet been written to. The UNWRITTEN flag in the corresponding data storage location field 326 (metadata extent record in the OCFS2 file system) will be set, which means that the storage unit 354 contains invalid data. When reading data from the storage unit 354, the file system directly returns a value of 0 to prevent the invalid data from being read.

[0107] When writing data to the storage unit 354, the amount of data that can be stored in the storage space of the storage unit 354 is obtained. When the amount of data to be written is less than the amount of data that can be stored in the storage space of the storage unit 354, the file system first allocates a bitmap object. Each bit in the bitmap object corresponds to the minimum modified data unit in the storage unit. The data amount of the above minimum data unit is the minimum modified data amount of the current file management system. Figure 3 The multiple storage subspaces in the bitmap object are the minimum data units. Only the bits corresponding to the minimum data unit where data is written are set, and the remaining bits in the bitmap object are 0. The specific steps are as follows:

[0108] First, execute Figure 7 In step S702, a new data structure is added.

[0109]

[0110] Among them, ocfs2_indi_bm_rec is a newly added data structure, ir_blkno is the first description information, pointing to the actual address of the storage unit (the second storage location), ir_count is the second description information, indicating the number of bits of the bitmap object, and ir_bm is the third description information, indicating the data bitmap. It can be understood that a data bitmap is an array, and there are multiple objects in an array. It should be noted that each object is 8 bits, and the bit of each array object corresponds to a minimum write unit.

[0111] Afterwards, when data is written, the file system allocates a storage unit for the data to be written. At this time, step S704 is executed to obtain the amount of data that can be stored in the storage unit.

[0112] Execute step S706 to determine whether the amount of data that can be stored in the storage unit is greater than the data to be written. If not, it means that the data object to be stored can occupy the entire space of the storage unit, and the invalid data previously existing in the storage unit will be overwritten by the data object. After that, when reading data, invalid data will not be read, so step S792 can be executed to write the data to be written into the storage unit.

[0113] If the judgment result of step S706 is yes, it means that the data object to be stored cannot occupy the entire space of the storage unit, and the invalid data previously existing in the storage unit cannot be completely overwritten by the data object. Later, when reading data, invalid data may be read. Therefore, it is necessary to execute step S708 to create a bitmap object, and avoid reading dirty data when reading data by adjusting the bit position of the bitmap object.

[0114] In the above step S708, Figure 3 The process of creating a bitmap object in the storage unit 354 includes adding an OCFS2_EXT_PARTIAL flag to the eflag of the data storage location field 326, which indicates that the data storage location field 326 points to a data structure rather than a storage unit. Figure 4 As shown, the data storage location field 326 no longer directly points to the storage unit 354, but points to the bitmap object 412 corresponding to the storage unit 354, and each bit of the bitmap object 412 points to a corresponding storage subspace.

[0115] It should be noted that when eflag = OCFS2_EXT_PARTIAL in the OCFS2 file system, it means that the current metadata extent record in the OCFS2 file system points to a data structure rather than a storage unit. When OCFS2_EXT_PARTIAL in eflag is not set, the extent record directly points to the data segment.

[0116] After the bitmap object 412 is created, the data written by the user into the storage unit 354 will directly fall into the storage subspace. At this time, step S712 is executed to set the bit corresponding to the storage subspace. Assume Figure 4 The storage subspace responsible for storing user-written data is storage subspace 344. At this time, the setting object of step S712 is bit 404 corresponding to storage subspace 344. The setting operation is performed on bit 404, and the other bits are 0.

[0117] When reading data from the storage unit where the bitmap object is created, the storage subspace corresponding to the unset bits will return a zero value, avoiding reading dirty data. By avoiding returning invalid data through the setting of the bitmap bits, there is no need to write a zero value when writing data, avoiding the write amplification problem of small blocks of data.

[0118] At the same time, a data regularization process is introduced, the main functions of which are as follows:

[0119] 1. Periodically scan the storage units in the file system whose data writing volume exceeds the preset threshold, and add the storage units whose data writing volume exceeds the preset threshold to the data regularization list.

[0120] 2. Based on the current system pressure, trigger the data consolidation task when the pressure is low. For example, when the total amount of data read but not written in the current file system is less than the preset data volume threshold Y, trigger the data consolidation task as follows:

[0121] (1) Write 0 to all unwritten areas of the storage units recorded in the data regularization table.

[0122] (2) Clear the OCFS2_EXT_PARTIAL flag in the eflag of the extent record corresponding to the storage unit that has completed writing all zeros to the unwritten area in the above step (1), and make the extent record directly point to the storage unit instead of pointing to the data structure ocfs2_indi_bm_rec.

[0123] (3) Reclaim the space of the data structure ocfs2_indi_bm_rec.

[0124] It should be noted that for the sake of simplicity, the aforementioned method embodiments are described as a series of actions. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, as certain steps may be performed in a different order or simultaneously. Furthermore, those skilled in the art should also be aware that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0125] According to another aspect of the embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 5 As shown, the device includes:

[0126] The information acquisition unit 502 is configured to acquire, in response to a data write operation, unit description information of a storage unit allocated to a data object to be written;

[0127] A bitmap creation unit 504 is configured to create a bitmap object that does not match the storage unit if the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit;

[0128] The setting unit 506 is used to perform a setting operation on at least one target bit in the bitmap object when writing a data object to at least one target storage subspace in multiple storage subspaces, wherein the at least one target bit is used to indicate the storage status of at least one target storage subspace.

[0129] Optionally, the above-mentioned data processing device also includes: a field update unit, used to obtain a first storage location of a data structure for storing a bitmap object; updating the data storage location field in the unit description information according to the first storage location, wherein the data storage location field before the update is used to indicate the second storage location of the storage unit; updating the target identification field in the unit description information, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the target identification field after the update is used to indicate that the data storage location field points to the data structure.

[0130] Optionally, the above-mentioned data processing device also includes: an information determination unit, used to: obtain the second storage position of the storage unit, and determine the second storage position as the first descriptive information in the data structure; determine the number of digits in the bitmap object as the second descriptive information in the data structure; determine the bitmap object as the third descriptive information in the data structure.

[0131] Optionally, the above-mentioned data processing device also includes: a writing unit, which is used to: write the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object; write the first data part of the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, and obtain the unit description information of the reference storage unit allocated to the second data part of the data object.

[0132] Optionally, the above-mentioned bitmap creation unit 504 includes one of the following: a first determination module, used to: determine the target quantity as the number of bits included in the bitmap object, wherein the target quantity is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data of the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data; a second determination module, used to obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

[0133] Optionally, the above-mentioned data processing device also includes: a data reading unit, used to: in response to a data reading operation, obtain unit description information of at least one storage unit for storing a data object; in a case where the target identification field in the unit description information indicates that the data storage location field points to a data structure, determine the data structure according to the storage location field; obtain a second storage location for indicating the storage unit from the data structure, and obtain a reference bitmap object from the data structure; in a case where at least one reference bit in the reference bitmap object is set, obtain at least one data object from the storage unit located at the second storage position according to the storage subspace corresponding to at least one bit.

[0134] Optionally, the above-mentioned data processing device also includes: a bitmap writing unit, which is used to: when the target identification field in the unit description information indicates that the data storage location field points to a data structure, allocate at least one reference storage subspace for the data object to be written from multiple storage subspaces, wherein the bit corresponding to the reference storage subspace is not set; when writing the data object to at least one reference storage subspace, perform a set operation on the bit corresponding to the non-reference storage subspace in the bitmap object.

[0135] Optionally, the above-mentioned data processing device also includes: a bitmap clearing unit, which is used to: when the current system load parameters meet the target conditions, determine the current storage unit to be processed from multiple storage units configured with their own matching bitmap objects; clear the storage space in the current storage unit that does not store valid data; clear the bitmap objects that do not match the current storage unit; and update the current unit description information of the current storage unit, wherein the updated current unit description information is used to indicate the storage status of the current storage unit.

[0136] According to another aspect of the embodiment of the present application, an electronic device for implementing the above data processing method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a mobile phone or a computer as an example. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0137] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0138] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0139] S1, in response to a data write operation, obtaining unit description information of a storage unit allocated for a data object to be written;

[0140] S2, when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, creating a bitmap object that does not match the storage unit, wherein multiple bits in the bitmap object are respectively used to indicate storage states of multiple storage subspaces in the storage unit;

[0141] S3. When writing a data object into at least one target storage subspace among a plurality of storage subspaces, a set operation is performed on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate a storage status of the at least one target storage subspace.

[0142] Alternatively, those skilled in the art will appreciate that Figure 6 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 5 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, etc.) shown in the Figure 5 Different configurations shown.

[0143] Among them, the memory 602 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for adjusting the read voltage in the memory in the embodiment of the present application. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, realizing the above-mentioned method for adjusting the read voltage in the memory. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include a memory set relative to the processor 604 process, and these process memories may be connected to the terminal via a network. Examples of the above-mentioned networks include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks and combinations thereof. Among them, the memory 602 can be used specifically but not limited to store information such as page elements and page styles. As an example, if Figure 6 As shown, the memory 602 may include, but is not limited to, the information acquisition unit 502, the bitmap creation unit 504, and the setting unit 506 in the data processing device. In addition, it may also include, but is not limited to, other module units in the read voltage adjustment device in the memory, which will not be repeated in this example.

[0144] Optionally, the transmission device 606 is configured to receive or transmit data via a network. Specific examples of the aforementioned network may include wired networks and wireless networks. In one embodiment, the transmission device 606 includes a network interface controller (NIC), which can be connected to other network devices and routers via a network cable to enable communication with the Internet or a local area network. In one embodiment, the transmission device 606 is a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0145] In addition, the electronic device further includes: a display 608 for displaying the target page; and a connection bus 610 for connecting various module components in the electronic device.

[0146] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting multiple nodes via network communication. The nodes may form a point-to-point network, and any computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the point-to-point network.

[0147] According to one aspect of the present application, a computer-readable storage medium is provided. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in the various optional implementations described above.

[0148] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0149] S1, in response to a data write operation, obtaining unit description information of a storage unit allocated for a data object to be written;

[0150] S2, when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, creating a bitmap object that does not match the storage unit, wherein multiple bits in the bitmap object are respectively used to indicate storage states of multiple storage subspaces in the storage unit;

[0151] S3. When writing a data object into at least one target storage subspace among a plurality of storage subspaces, a set operation is performed on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate a storage status of the at least one target storage subspace.

[0152] Optionally, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0153] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0154] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above-mentioned computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0155] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is merely a logical functional division. In actual implementation, there may be other division methods, such as combining or integrating multiple units or components into another system, or ignoring or not implementing some features. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interface, indirect coupling or communication connection of units or modules, and may be electrical or other forms.

[0157] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0158] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0159] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be considered as the scope of protection of the present application.

[0160] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0161] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above method embodiments when run.

[0162] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0163] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0164] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0165] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0166] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above method embodiments are implemented.

[0167] An embodiment of the present application also provides a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above method embodiments.

[0168] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0169] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0170] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: In response to a data write operation, obtaining unit description information of a storage unit allocated for a data object to be written; When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, creating a bitmap object matching the storage unit, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit; When writing the data object to at least one target storage subspace among the plurality of storage subspaces, performing a setting operation on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate a storage state of the at least one target storage subspace; After obtaining the unit description information of the storage unit allocated for the data object to be written, the method further includes: When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object, writing the data object into the remaining storage space of the storage unit; When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, the first data part of the data object is written in the remaining storage space of the storage unit, and the unit description information of the reference storage unit allocated to the second data part of the data object is obtained.

2. The method according to claim 1, characterized in that After creating the bitmap object matching the storage unit, the method further includes: Obtaining a first storage location of a data structure for storing the bitmap object; updating a data storage location field in the unit description information according to the first storage location, wherein the data storage location field before the update is used to indicate a second storage location of the storage unit; The target identification field in the unit description information is updated, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the target identification field after the update is used to indicate that the data storage location field points to the data structure.

3. The method according to claim 2, characterized in that Before obtaining the storage location of the data structure for storing the bitmap object, the method further includes: Obtaining the second storage location of the storage unit, and determining the second storage location as the first description information in the data structure; Determine the number of digits in the bitmap object as second description information in the data structure; The bitmap object is determined as the third description information in the data structure.

4. The method according to claim 2, characterized in that The creating of a bitmap object matching the storage unit includes one of the following: Determining a target number as the number of bits included in the bitmap object, wherein the target number is a ratio between a total amount of data that can be stored in the storage unit and a minimum amount of modified data in the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data; Obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

5. The method according to any one of claims 1 to 4, characterized in that After performing a setting operation on at least one target bit in the bitmap object in a case where the data object is written into at least one target residual subspace among the plurality of residual subspaces, the method further comprises: In response to a data read operation, obtaining unit description information of at least one of the storage units for storing a data object; In a case where the target identification field in the unit description information indicates that the data storage location field points to a data structure, determining the data structure according to the data storage location field; Obtaining a second storage location indicating the storage unit from the data structure, and obtaining a reference bitmap object from the data structure; In a case where at least one reference bit in the reference bitmap object is set, at least one data object is obtained from the storage unit located at the second storage position according to the storage subspace corresponding to the at least one reference bit.

6. The method according to any one of claims 1 to 4, characterized in that After performing the setting operation on at least one target bit in the bitmap object, the method further includes: In the case where the current system load parameter meets the target condition, determining a current storage unit to be processed from the plurality of storage units configured with the respective matching bitmap objects; Clearing the storage space in the current storage unit that does not store valid data; Clearing the bitmap object matching the current storage unit; The current unit description information of the current storage unit is updated, wherein the updated current unit description information is used to indicate the storage status of the current storage unit.

7. A data processing device, characterized in that: include: an information acquisition unit, configured to acquire, in response to a data writing operation, unit description information of a storage unit allocated for a data object to be written; a bitmap creation unit, configured to create a bitmap object matching the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit; a setting unit, configured to, when writing the data object into at least one target storage subspace among the plurality of storage subspaces, perform a setting operation on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate a storage state of the at least one target storage subspace; The data processing device is further used to: write the data object into the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object; write the first data part of the data object into the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, and obtain the unit description information of the reference storage unit allocated to the second data part of the data object.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 6 when executed by a processor.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images