Data processing method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a data processing method and device, a storage medium and electronic equipment. The method comprises the following steps: in response to a data writing operation, acquiring unit description information of a storage unit distributed for a to-be-written data object; under the condition that the unit description information indicates that the reference data volume capable of being stored in the residual storage space of the storage unit is larger than the first data volume of the data object, a bitmap object matched with the storage unit is created, and a plurality of bits in the bitmap object are used for indicating the storage states of a plurality of storage subspaces in the storage unit respectively; under the condition that the data object is written into at least one target storage subspace in the multiple storage subspaces, setting operation is carried out on at least one target bit in the bitmap object, and the at least one target bit is used for indicating the storage state of the at least one target storage subspace.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a data processing method and device, a storage medium, and an electronic device. Background Art

[0002] In cloud computing and virtualization application scenarios, the file system is the core technology for building storage pools. The file system divides the backend data disk into data blocks (metadata) of a certain size to perform space management and data organization. The increasing storage pools in cloud computing scenarios and the increasing number of large virtual disk application scenarios pose a great challenge to the metadata management of the file system.

[0003] However, in the existing file system technology, the backend storage space allocated by the file system to the virtual machine will not be initialized when the data is allocated. Therefore, the backend storage space of the virtual machine will contain a large amount of invalid data. When the virtual machine writes data to a data block for the first time, in order to avoid reading invalid data in the data block later, the file system will perform a zero operation on the part of the data block that does not store the written data. That is to say, for the memory, in addition to writing the target data that needs to be written, several zero values ​​are also written. This process is also called write amplification. That is to say, in the related technology, there is a problem of write amplification in the process of writing data.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a data processing method and device, a storage medium, and an electronic device to at least solve the technical problem of write amplification caused by data writing existing in the related art.

[0006] According to an embodiment of the present application, there is provided a data processing method, comprising: in response to a data writing operation, acquiring unit description information of a storage unit allocated for a data object to be written;

[0007] When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created, wherein multiple bits in the bitmap object are respectively used to indicate the storage status of multiple storage subspaces in the storage unit; when the data object is written to at least one target storage subspace among the multiple storage subspaces, a set operation is performed on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate the storage status of at least one target storage subspace.

[0008] According to another aspect of an embodiment of the present application, there is further provided a data processing device, comprising: an information acquisition unit, configured to acquire, in response to a data writing operation, unit description information of a storage unit allocated to a data object to be written;

[0009] A bitmap creation unit, configured to create a bitmap object that does not match the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit;

[0010] A setting unit is used to perform a setting operation on at least one target bit in a bitmap object when writing a data object to at least one target storage subspace among multiple storage subspaces, wherein the at least one target bit is used to indicate the storage status of at least one target storage subspace.

[0011] Optionally, the above-mentioned data processing device also includes: a field updating unit, used to obtain a first storage location of a data structure for storing a bitmap object; updating the data storage location field in the unit description information according to the first storage location, wherein the data storage location field before the update is used to indicate the second storage location of the storage unit; updating the target identification field in the unit description information, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the updated target identification field is used to indicate that the data storage location field points to the data structure.

[0012] Optionally, the above-mentioned data processing device also includes: an information determination unit, used to: obtain the second storage position of the storage unit, and determine the second storage position as the first description information in the data structure; determine the number of digits in the bitmap object as the second description information in the data structure; determine the bitmap object as the third description information in the data structure.

[0013] Optionally, the above-mentioned data processing device also includes: a writing unit, used to: write the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object; write the first data part of the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, and obtain the unit description information of the reference storage unit allocated to the second data part of the data object.

[0014] Optionally, the above-mentioned bitmap creation unit includes one of the following: a first determination module, used to: determine the target number as the number of bits included in the bitmap object, wherein the target number is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data of the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data; a second determination module, used to obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

[0015] Optionally, the above-mentioned data processing device also includes: a data reading unit, used to: in response to a data reading operation, obtain unit description information of at least one storage unit used to store a data object; when the target identification field in the unit description information indicates that the data storage location field points to a data structure, determine the data structure according to the storage location field; obtain a second storage location used to indicate the storage unit from the data structure, and obtain a reference bitmap object from the data structure; when at least one reference bit in the reference bitmap object is set, obtain at least one data object from the storage unit located at the second storage position according to the storage subspace corresponding to at least one bit.

[0016] Optionally, the above-mentioned data processing device also includes: a bitmap writing unit, which is used to: when the target identification field in the unit description information indicates that the data storage location field points to a data structure, allocate at least one reference storage subspace for the data object to be written from multiple storage subspaces, wherein the bit corresponding to the reference storage subspace is not set; when writing the data object to at least one reference storage subspace, perform a setting operation on the bit corresponding to the non-reference storage subspace in the bitmap object.

[0017] Optionally, the above-mentioned data processing device also includes: a bitmap clearing unit, which is used to: when the current system load parameters meet the target conditions, determine the current storage unit to be processed from a plurality of storage units configured with respective matching bitmap objects; clear the storage space in the current storage unit that does not store valid data; clear the bitmap objects that do not match the current storage unit; and update the current unit description information of the current storage unit, wherein the updated current unit description information is used to indicate the storage status of the current storage unit.

[0018] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above data processing method.

[0019] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the data processing method through the computer program.

[0020] Through the present application, first, in response to a data write operation, the unit description information of the storage unit allocated to the data object to be written is obtained. When the unit description information indicates that the reference data amount that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created, and each bit in the bitmap object corresponds to a storage subspace in the storage unit, thereby achieving the determination of the storage status of multiple storage subspaces in the storage unit through the setting conditions of multiple bits in the bitmap object. Specifically, when the data object is written to the storage subspace in the storage unit, the corresponding bit of the storage subspace will be set, while the bit of the storage subspace where the data is not written is not set. Then, in the process of reading data, the file system can return a value of 0 according to the setting condition of the target bit when the target bit is not set, without having to avoid reading invalid data by setting the part of the storage unit that does not store the data to be written to zero. In this scheme, when the amount of data to be written is smaller than the amount of data in the storage unit, a bitmap object that does not match the storage unit is created. Only the data to be written needs to be written to the storage unit, and there is no need to write a 0 value to the storage unit to avoid reading invalid data. No data other than the data to be written will be written. In other words, the technical problem of write amplification accompanying data writing in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 It is a hardware structure block diagram of a server device of a data processing method according to an embodiment of the present application;

[0023] Figure 2 is a flow chart of a data processing method according to an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of a data processing method according to an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of another data processing method according to an embodiment of the present application;

[0026] Figure 5 is a structural schematic diagram of a data processing device according to an embodiment of the present application;

[0027] Figure 6 is a structural schematic diagram of a data processing electronic device according to an embodiment of the present application;

[0028] Figure 7 is a flowchart of another data processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0031] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a data processing method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 The server device may include a processor 102 (only one is shown in the figure) (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components than shown, or with different Figure 1 Different configurations are shown.

[0032] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a method for adjusting a read voltage in a memory in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory arranged relative to the processor 102 process, and these process memories may be connected to a server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] As an optional implementation, Figure 2 As shown, the above data processing method includes:

[0035] S202, in response to a data writing operation, obtaining unit description information of a storage unit allocated for a data object to be written;

[0036] S204, when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, create a bitmap object that does not match the storage unit, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit;

[0037] S206, when writing the data object into at least one target storage subspace among the multiple storage subspaces, performing a set operation on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate a storage state of the at least one target storage subspace.

[0038] It should be noted that the above-mentioned storage unit is the smallest unit for storing file data in the file system. When writing data, the above-mentioned step S202 is triggered: in response to the data writing operation, the unit description information of the storage unit allocated for the data object to be written is obtained. It should be noted that the above-mentioned storage unit allocated for the data object to be written is a storage unit allocated by the file system for the data to be written when there is data to be written, and is used to store the data to be written. In addition, the above-mentioned description information includes the amount of data that can be stored in the remaining storage space of the above-mentioned storage unit;

[0039] Then, step S204 is executed, in the case where the reference data amount that can be stored in the remaining storage space of the storage unit indicated by the unit description information is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created, wherein a plurality of bits in the bitmap object are respectively used to indicate the storage status of a plurality of storage subspaces in the storage unit;

[0040] It should be noted that the above bitmap object includes multiple bits, each bit corresponds to a storage subspace in the storage unit, and the above operation of creating a bitmap object that does not match the storage unit includes one of the following two methods:

[0041] Method 1: Determine the target number as the number of bits included in the bitmap object, wherein the target number is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data in the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data;

[0042] Method 2: Obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

[0043] In the above-mentioned method 1, the ratio of the total amount of data that can be stored in the storage unit to the minimum amount of modified data in the current file management system is used as the number of bits of the above-mentioned bitmap object. In the above-mentioned method 2, the ratio of the total amount of data that can be stored in the storage unit to the first amount of data of the data object to be stored is used as the number of bits of the above-mentioned bitmap object.

[0044] In an optional embodiment, when the first data volume of the above-mentioned data object is less than a preset threshold, the ratio between the total data volume that can be stored in the storage unit and the minimum modified data volume of the current file management system is obtained, and the ratio is used as the number of bits of the above-mentioned bitmap object.

[0045] In another optional embodiment, when the first data amount of the above-mentioned data object is greater than a preset threshold, the ratio between the total data amount that can be stored in the storage unit and the first data amount of the data object is obtained, and the ratio between the total data amount and the first data amount is further determined as the number of bits included in the bitmap object.

[0046] Through the above two methods, a data bitmap containing multiple bits can be determined, and each bit can correspond to a storage space in the storage unit. It can be understood that, for the above method one, the size of the storage subspace corresponding to each bit is the minimum data modification amount of the current file management system, and for the above method two, the size of the storage subspace corresponding to each bit is the size of the data to be written, thereby realizing the dynamic determination of the number of bits of the bitmap, so that in the process of bitmap creation, a scheme with a small number of bit generation can be dynamically selected according to the size of the data to be written, thereby reducing the overhead of the file system and improving data processing efficiency.

[0047] Then, the above step S208 can be executed, and when the data object is written to at least one target storage subspace in the multiple storage subspaces, a setting operation is performed on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate the storage status of at least one target storage subspace.

[0048] It should be noted that the above-mentioned set operation is to set the value of the target bit to 1. It can be understood that the value of the above-mentioned target bit before performing the set operation is 1. Each target bit is used to indicate the storage status of the corresponding target storage subspace. When the value of the target bit is 1, it indicates that the corresponding target subspace stores write data. When the value of the target bit is 0, it indicates that the corresponding target subspace does not store write data. It should be noted that the target subspace does not store write data is not equivalent to no data storage. In fact, the target subspace may store invalid data. In the case of storing invalid data, the corresponding target bit is not set and the value is 0. After the data to be written is written into the target subspace, the target subspace stores the write data. In this case, the target bit is set and the value is 1.

[0049] Through the above method of the present application, first, in response to a data write operation, the unit description information of the storage unit allocated to the data object to be written is obtained, and when the unit description information indicates that the reference data amount that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, a bitmap object that does not match the storage unit is created, wherein multiple bits in the bitmap object are respectively used to indicate the storage status of multiple storage subspaces in the storage unit; when the data object is written to at least one target storage subspace among the multiple storage subspaces, a setting operation is performed on at least one target bit in the bitmap object, that is, the target bit of the storage subspace where the data is not written is not set, thereby enabling the file system to return a value of 0 according to the setting status of the target bit when the target bit is not set, without having to avoid reading dirty data by setting the part of the storage unit that does not store the data to be written to zero. This solution, which only needs to write the data to be written to the memory without having to write a value of 0 additionally, solves the technical problem of write amplification when writing data in the prior art.

[0050] As an optional implementation, after creating a bitmap object that does not match the storage unit, the method further includes:

[0051] S1, obtaining a first storage location of a data structure for storing a bitmap object;

[0052] S2, updating the data storage location field in the unit description information according to the first storage location, wherein the data storage location field before the update is used to indicate the second storage location of the storage unit;

[0053] S3, updating the target identification field in the unit description information, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the target identification field after the update is used to indicate that the data storage location field points to the data structure.

[0054] It should be noted that, in the above step S1, the data structure of the bitmap object refers to a custom data type including a bitmap object field. For example, in an optional implementation, there is the following data structure:

[0055]

[0056] The above ocfs2_indi_bm_rec is a data structure, which is a user-defined C++ data type, and ir_bm is a bitmap object variable.

[0057] In an optional embodiment, after executing the above step S1 and obtaining the storage location of the data structure, execute the above step S2 to update the data storage location field of the above unit description information, so that the data location storage field is changed from indicating the storage location of the storage unit to indicating the storage location of the data structure.

[0058] Then, step S3 is executed to update the target representation field in the unit description information. It should be noted that the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and after the above step S2 is executed, the above data storage location field is changed from indicating the storage location of the storage unit to indicating the storage location of the data structure, that is, after the above step S2 is executed, the above data storage location field is changed from pointing to the storage unit to pointing to the data structure, so it is necessary to update the target representation field pointed to by the data storage location field, so that the updated target representation field indicates that the above data storage location field points to the data structure.

[0059] The following combination Figure 3 , Figure 4 The above steps S1-S3 are explained.

[0060] Figure 3 This is a schematic diagram of the underlying file data organization of a file system. Figure 3 In the file system, a file data mainly includes identification information 302, which is used to record key information of the file, such as the size of the file, the creation time of the file, etc. A file data also includes a data node tree composed of multiple data nodes, and multiple storage units. The storage unit is the smallest unit for recording user data. In the file system, a file will be composed of multiple discrete storage units with different lengths.

[0061] Figure 3 In the example, the data storage location field 324 points to the storage unit 352, and the data storage location field 326 points to the storage unit 354. After the bitmap object is created for the storage unit 354, the data storage location field 326 is changed from indicating the storage location of the storage unit to indicating the storage location of the structure of the bitmap object. Figure 4 As shown, the data storage location field 326 no longer directly points to the storage unit 354, but points to the bitmap object 412 corresponding to the storage unit 354, and each bit of the bitmap object 412 points to a corresponding storage subspace. In addition, there is a target identification field (not shown in the figure) in the file system. The target identification field is used to indicate that the data storage location field 326 points to the storage unit 354 in the initial state, and is updated after the bitmap object 412 is created. The updated target identification field indicates that the data storage location field 326 points to the bitmap object 412.

[0062] Through the above-mentioned implementation mode of the present application, the first storage position of the data structure for storing the bitmap object is first obtained, and then the data storage location field in the unit description information is updated according to the first storage position, and the target identification field in the unit description information is updated so that the updated target identification field indicates that the data storage location field points to the data structure.

[0063] As an optional implementation, before obtaining the storage location of the data structure for storing the bitmap object, the method further includes:

[0064] S1, obtaining a second storage location of a storage unit, and determining the second storage location as first description information in a data structure;

[0065] S2, determining the number of digits in the bitmap object as second description information in the data structure;

[0066] S3, determining the bitmap object as the third description information in the data structure.

[0067] It should be noted that the first description information, the second description information, and the third description information determined in the above steps S1 to S3 are three variables of the above data structure, for example, the above data structure may be in the following form:

[0068]

[0069] Among them, ir_blkno is the first description information, pointing to the actual address of the storage unit (the second storage location), ir_count is the second description information, indicating the number of bits of the bitmap object, and ir_bm is the third description information, indicating the data bitmap. It can be understood that a data bitmap is an array, and there are multiple objects in an array. It should be noted that each object is 8 bits, and the bit of each array object corresponds to a minimum write unit.

[0070] As an optional implementation manner, after obtaining the unit description information of the storage unit allocated for the data object to be written, the method further includes:

[0071] S1, writing the data object into the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object;

[0072] S2. When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, the first data portion of the data object is written into the remaining storage space of the storage unit, and the unit description information of the reference storage unit allocated to the second data portion of the data object is obtained.

[0073] It can be understood that after obtaining the description information of the storage unit allocated to the data object to be written, the data object to be written will choose to write one or more storage units according to its own data volume. When the amount of data that can be stored in the above storage unit is equal to the amount of data of the data object to be written, the data object to be written will be written into the remaining writable storage space in the above storage unit that does not store other write data. When the amount of data of the data object to be written is greater than the amount of data that can be stored in the above storage unit, the data object to be written will store a part of its own data in the storage unit according to the amount of data that can be stored in the storage unit, and for the unstored part, a new storage unit will be allocated, and the description information of the new storage unit will be obtained, and the unstored part will be stored according to the storable data volume indicated by the description information of the new storage unit. If there is still an unstored part, it will continue to iterate until all the data to be written are stored or there is no storage unit in the file system that can store new data. The following is an example to explain.

[0074] For example, in an optional implementation, assume that the data object A to be written currently has 5 units of length of data, and the file system allocates storage unit A for the data to be written currently, and storage unit A has two units of length of data storage space. In this case, the two units of length of data object A to be written are stored in storage unit A as the first data part, and the remaining three units of length of data are used as a new data object B to be written (the second data part), and a storage unit B is continuously allocated to the new data object B to be written, and storage unit B has two units of length of data storage space. In this case, the two units of length of data object B are stored in storage unit B as the first data part, and the remaining one unit of length of data is used as a new data object C to be written, and a storage unit C is continuously allocated to data object C, and storage unit C has two units of length of data storage space. Unit length of storable data space, in this case, the data object C is completely stored in the storage unit C, and the iteration stops at this time. At the same time, the bitmap object creation operation of the storage unit C is triggered to create a bitmap object C corresponding to the storage unit C. It is assumed that the number of bits of the bitmap object C at this time is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data of the current file management system. When the minimum amount of modified data is half of the above unit length, it is determined that the bitmap object C has four bits, and each bit corresponds to a storage subspace in the storage unit C. It can be understood that the above data object C is stored in a storage subspace in the storage unit C, and the bit corresponding to the storage subspace will be set to indicate that valid data is stored in the storage subspace, while other bits that are not set indicate that no data is stored in the corresponding storage subspace or invalid data is stored.

[0075] Through the above-mentioned implementation mode of the present application, when the amount of data to be written currently exceeds the amount of data that can be stored in the allocated storage unit, the data to be written can be stored in multiple storage units through a loop iteration method. While the data object to be written currently can be completely stored, during the last iteration, when the amount of data in the allocated storage unit is greater than the amount of data to be written in this iteration, the creation of the bitmap can be triggered, thereby reducing the operating overhead of the file system and improving the operating efficiency.

[0076] As an optional implementation, in the case of writing the data object into at least one target remaining subspace of the plurality of remaining subspaces, after performing a setting operation on at least one target bit in the bitmap object, the method further includes:

[0077] S1, in response to a data read operation, obtaining unit description information of at least one storage unit for storing a data object;

[0078] S2-1, when the target identification field in the unit description information indicates that the data storage location field points to the storage unit, and the unit description information indicates that the storage unit stores the first reference data object that has been written, determining the first reference data object in the storage unit as the data object;

[0079] S2-2, when the target identification field in the unit description information indicates that the data storage location field points to the storage unit, and the unit description information indicates that the first reference data object to be written is not stored in the storage unit, a zero value is determined as the data object;

[0080] S2-3, when the target identification field in the unit description information indicates that the data storage location field points to a data structure, determining the data structure according to the storage location field;

[0081] S3, obtaining a second storage location indicating the storage unit from the data structure, and obtaining a reference bitmap object from the data structure;

[0082] S4, when at least one reference bit in the reference bitmap object is set, obtaining at least one data object from a storage unit located at a second storage position according to the storage subspace corresponding to at least one of the bits.

[0083] It should be noted that the bitmap object is created when the data amount of the data object to be written is less than the storable data amount of the storage unit. That is to say, when the data amount of the data object to be written is equal to the storable data amount of the storage unit, the bitmap object will not be created. Therefore, in the above step S1, the storage unit of the above data object to be read may have a corresponding bitmap object or may not have a corresponding bitmap object.

[0084] It should be further explained that, when part of the storage space of the storage unit stores the data written by the user, there is a corresponding bitmap object in the storage unit. At this time, the data to be read is determined according to the setting status of each bit of the bitmap object. When the bit is set, it means that the storage subspace corresponding to the bit stores the data written by the user. At this time, the data in the storage subspace is determined as the data to be read. When the bit of the bitmap object is not set, it means that the storage subspace corresponding to the bit stores the data written by the user. At this time, invalid data may be stored in the storage subspace. Therefore, the data in the storage subspace cannot be used as the data to be read. Instead, the zero value is used as the data to be read.

[0085] In addition, when a storage unit does not have a corresponding bitmap object, it means that all storage spaces of the storage unit have stored the data written by the user, or all storage spaces of the storage unit have not stored the data written by the user. When all storage spaces have stored the data written by the user, the storage data in the storage unit is used as the data to be read. When all storage spaces of the storage unit have not stored the data written by the user, invalid data may be stored in the above storage unit. Therefore, the data in the storage subspace cannot be used as the data to be read at this time, but the zero value is used as the data to be read.

[0086] The above steps S1 to S4 are described below by way of example:

[0087] First, obtain the unit description information of at least one storage unit that stores the data object to be read. Assuming that data needs to be read from three storage units A, B, and C at this time, obtain the unit description information corresponding to each of the storage units A, B, and C. Assuming that the unit description information of storage unit A indicates that storage unit A has no corresponding bitmap object, and the data in storage unit A is data written by the user, it means that all the data in storage unit A is data written by the user. At this time, all the data in storage unit A (the first data object) is used as the data object.

[0088] Assume that the unit description information of storage unit B indicates that storage unit A has no corresponding bitmap object, and the data in storage unit B is not data written by the user, and in this case, a zero value is used as the data object.

[0089] Assume that the target identification field in the unit description information of the storage unit C indicates that the data storage location field points to the data structure. In this case, the data storage location field no longer points to the storage unit C, but points to the data structure. Further, the second storage location indicating the storage unit C is obtained from the data structure, and the data structure is instantiated, and the bitmap variable in the instantiated data structure is called to generate the storage unit C. At this time, the target reference data object is determined according to the setting of each target bit of the bitmap object C. Specifically, it is assumed that there are two target bits in the bitmap object C: target bit A and target bit B. Assuming that the target bit A is set and the target bit B is not set, the data stored in the storage subspace corresponding to the target bit A is used as the data object corresponding to the target bit A, and the zero value is used as the data object corresponding to the target bit B. Thus, the read operation of the above data object is realized.

[0090] Through the above-mentioned implementation mode of the present application, data can be read according to the data storage situation of the storage unit and the corresponding bit position situation. When the storage unit completely stores the data written by the user, the data in the storage unit is returned. When the storage unit does not store the data written by the user, a value of 0 is returned. When part of the storage subspace in the storage unit stores the data written by the user, data is returned according to the setting situation of each bit of the bitmap of the storage unit. If the bit corresponding to the storage subspace where the data to be read is located is set, the data in the storage subspace is returned, and when the bit corresponding to the storage subspace where the data to be read is located is not set, a value of 0 is returned, so that there is no need to avoid reading dirty data by setting the part of the storage unit that does not store the user-written data to zero, which solves the technical problem of write amplification when writing data in the prior art.

[0091] As an optional implementation manner, in response to the data writing operation, after obtaining the unit description information of the storage unit allocated for the data object to be written, the method further includes:

[0092] S1, when the target identification field in the unit description information indicates that the data storage location field points to a data structure, at least one reference storage subspace is allocated for the data object to be written from multiple storage subspaces, wherein the target bit corresponding to the reference storage subspace is not set;

[0093] S2: When writing the data object into at least one reference storage subspace, performing a setting operation on a target bit corresponding to the non-reference storage subspace in the bitmap object.

[0094] It can be understood that in the above step S1, when the storage unit allocated to the storage unit is an allocated bitmap, the target identification field in the unit description information of the above storage unit indicates that the data storage location field points to the data structure. At this time, according to the bit position of the bitmap object corresponding to the storage unit, a storage subspace is further allocated for the data object to be written.

[0095] It can be understood that the set bit in the bitmap object represents that the storage subspace corresponding to the bit has stored valid data, while the unset bit represents that the corresponding storage subspace does not store valid data. At this time, the storage subspace corresponding to the unset bit can be allocated to the data object to be written to store the data object to be written. When the data object to be stored is written into the allocated storage subspace, the bit corresponding to the storage subspace of the written data object is set.

[0096] Through the above-mentioned implementation mode of the present application, when the allocated storage unit is a storage unit for which a bitmap has been created, the storage subspace corresponding to the unset bit is allocated to the data object to be written, and the allocated storage subspace is set. Through the above-mentioned implementation mode of the present application, the data writing operation on the storage unit for which a bitmap has been created can be realized.

[0097] As an optional implementation, after performing a setting operation on at least one target bit in the bitmap object, the method further includes:

[0098] S1, when the current system load parameter meets the target condition, determining a current storage unit to be processed from a plurality of storage units configured with respective matching bitmap objects;

[0099] S2, clearing the storage space in the current storage unit that does not store valid data;

[0100] S3, clear the bitmap objects that do not match the current storage unit;

[0101] S4, updating the current unit description information of the current storage unit, wherein the updated current unit description information is used to indicate the storage status of the current storage unit.

[0102] It should be noted that the bitmap object itself is an index of the file system. The more bitmap objects are created, the greater the index overhead of the file system. Therefore, for the storage unit of the created bitmap object, when the parameters of the current system meet the target conditions, the storage subspace corresponding to the unset bits in the storage unit, that is, the storage space in the above step S2 that does not store valid data, is cleared. Specifically, the operation of writing a value of 0 is performed on the above storage space that does not store valid data.

[0103] When writing the value 0 is completed, there is no invalid data in the storage unit, and therefore, no invalid data will be read from the storage unit. Therefore, there is no need to determine the read data by whether the bit in the bitmap is set. At this time, the above step S3 can be executed to clear the bitmap object corresponding to the current storage space, and then step S4 can be executed to update the current unit description information of the current storage unit, so that the updated current unit description information indicates that no invalid data is stored in the current storage unit.

[0104] Through the above-mentioned implementation mode of the present application, when the current system load parameters meet the target conditions, the current storage unit to be processed is determined from multiple storage units configured with respective matching bitmap objects. Further, the storage space in the current storage unit that does not store valid data is cleared to make the current storage unit free of invalid data, and the bitmap object can be cleared to reduce the index overhead of the current system.

[0105] The following combination Figure 3 , Figure 4 , Figure 7 A complete data processing method is described.

[0106] Assume that the current file system is OCFS2 file system, Figure 3 The storage unit 354 in the file is a storage unit that has been allocated backend storage space but has not yet performed data writing. The UNWRITTEN flag in the corresponding data storage location field 326 (metadata extent record in the OCFS2 file system) will be set, which means that the storage unit 354 contains invalid data. When reading the data in the storage unit 354, the file system directly returns a value of 0 to prevent the invalid data from being read.

[0107] When writing data to the storage unit 354, the amount of data that can be stored in the storage space of the storage unit 354 is obtained. When the amount of data written is less than the amount of data that can be stored in the storage space of the storage unit 354, the file system first allocates a bitmap object. Each bit in the bitmap object corresponds to the minimum modified data unit in the storage unit. The data amount of the above minimum data unit is the minimum modified data amount of the current file management system. Figure 3 The multiple storage subspaces in the bitmap object are the smallest data units. Only the bit corresponding to the smallest data unit where data is written is set, and the remaining bits in the bitmap object are 0. The specific steps are as follows:

[0108] First, execute Figure 7 In step S702, a new data structure is added.

[0109]

[0110] Among them, ocfs2_indi_bm_rec is a newly added data structure, ir_blkno is the first description information, pointing to the actual address of the storage unit (the second storage location), ir_count is the second description information, indicating the number of bits of the bitmap object, and ir_bm is the third description information, indicating the data bitmap. It can be understood that a data bitmap is an array, and there are multiple objects in an array. It should be noted that each object is 8 bits, and the bit of each array object corresponds to a minimum write unit.

[0111] Afterwards, when writing data, the file system will allocate a storage unit for the data to be written. At this time, step S704 is executed to obtain the amount of data that can be stored in the storage unit.

[0112] Execute step S706 to determine whether the amount of data that can be stored in the storage unit is greater than the data to be written. If not, it means that the data object to be stored can occupy the entire space of the storage unit, and the invalid data previously existing in the storage unit will be overwritten by the data object. After that, when reading data, invalid data will not be read, so step S792 can be executed to write the data to be written into the storage unit.

[0113] If the judgment result of step S706 is yes, it means that the data object to be stored cannot occupy the entire space of the storage unit, and the invalid data previously existing in the storage unit cannot be completely overwritten by the data object. Later, when reading data, invalid data may be read. Therefore, it is necessary to execute step S708 to create a bitmap object, and avoid reading dirty data when reading data by adjusting the bit position of the bitmap object.

[0114] In the above step S708, Figure 3 The process of creating a bitmap object in the storage unit 354 in the example includes adding an OCFS2_EXT_PARTIAL flag to the eflag of the data storage location field 326, which indicates that the data storage location field 326 points to a data structure instead of a storage unit. Figure 4 As shown, the data storage location field 326 no longer directly points to the storage unit 354, but points to the bitmap object 412 corresponding to the storage unit 354, and each bit of the bitmap object 412 points to a corresponding storage subspace.

[0115] It should be noted that when eflag=OCFS2_EXT_PARTIAL in the OCFS2 file system, it means that the current metadata extent record in the OCFS2 file system points to a data structure rather than a storage unit. When OCFS2_EXT_PARTIAL in eflag is not set, the extent record directly points to the data segment.

[0116] After the bitmap object 412 is created, the data written by the user into the storage unit 354 will directly fall into the storage subspace. At this time, step S712 is executed to set the bit corresponding to the storage subspace. Assume Figure 4 The storage subspace responsible for storing the data written by the user is the storage subspace 344. At this time, the setting object of step S712 is the bit 404 corresponding to the storage subspace 344. The setting operation is performed on the bit 404, and the other bits are 0.

[0117] When reading data from the storage unit where the bitmap object is created, the storage subspace corresponding to the unset bit will return a zero value to avoid reading dirty data. In addition, by avoiding returning invalid data through the setting of the bitmap bit, there is no need to write a 0 value when writing data, avoiding the write amplification problem of small blocks of data.

[0118] At the same time, the data regularization process is introduced. The main functions of this process are as follows:

[0119] 1. Periodically scan the storage units in the file system whose data writing volume exceeds the preset threshold, and add the storage units whose data writing volume exceeds the preset threshold to the data regularization list.

[0120] 2. According to the current system pressure state, the data regularization task is triggered when the pressure is low. For example, when the total amount of data read but not written in the current file system is less than the preset data volume threshold Y, the data regularization task is triggered as follows:

[0121] (1) Write all the unwritten areas in the storage units recorded in the data regularization list to 0.

[0122] (2) Clear the OCFS2_EXT_PARTIAL flag in the eflag of the extent record corresponding to the storage unit that has completed writing all zeros to the unwritten area in the above step (1), and make the extent record directly point to the storage unit instead of pointing to the data structure ocfs2_indi_bm_rec.

[0123] (3) Reclaim the space of the data structure ocfs2_indi_bm_rec.

[0124] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0125] According to another aspect of the embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 5 As shown, the device comprises:

[0126] The information acquisition unit 502 is used to acquire unit description information of the storage unit allocated for the data object to be written in response to the data writing operation;

[0127] A bitmap creation unit 504 is used to create a bitmap object that does not match the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, wherein a plurality of bits in the bitmap object are respectively used to indicate the storage status of a plurality of storage subspaces in the storage unit;

[0128] The setting unit 506 is used to perform a setting operation on at least one target bit in the bitmap object when writing the data object to at least one target storage subspace in multiple storage subspaces, wherein the at least one target bit is used to indicate the storage status of at least one target storage subspace.

[0129] Optionally, the above-mentioned data processing device also includes: a field updating unit, used to obtain a first storage location of a data structure for storing a bitmap object; updating the data storage location field in the unit description information according to the first storage location, wherein the data storage location field before the update is used to indicate the second storage location of the storage unit; updating the target identification field in the unit description information, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the updated target identification field is used to indicate that the data storage location field points to the data structure.

[0130] Optionally, the above-mentioned data processing device also includes: an information determination unit, used to: obtain the second storage position of the storage unit, and determine the second storage position as the first description information in the data structure; determine the number of digits in the bitmap object as the second description information in the data structure; determine the bitmap object as the third description information in the data structure.

[0131] Optionally, the above-mentioned data processing device also includes: a writing unit, used to: write the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first data amount of the data object; write the first data part of the data object in the remaining storage space of the storage unit when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, and obtain the unit description information of the reference storage unit allocated to the second data part of the data object.

[0132] Optionally, the bitmap creation unit 504 includes one of the following: a first determination module, used to: determine the target number as the number of bits included in the bitmap object, wherein the target number is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data of the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data; a second determination module, used to obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

[0133] Optionally, the above-mentioned data processing device also includes: a data reading unit, used to: in response to a data reading operation, obtain unit description information of at least one storage unit used to store a data object; when the target identification field in the unit description information indicates that the data storage location field points to a data structure, determine the data structure according to the storage location field; obtain a second storage location used to indicate the storage unit from the data structure, and obtain a reference bitmap object from the data structure; when at least one reference bit in the reference bitmap object is set, obtain at least one data object from the storage unit located at the second storage position according to the storage subspace corresponding to at least one bit.

[0134] Optionally, the above-mentioned data processing device also includes: a bitmap writing unit, which is used to: when the target identification field in the unit description information indicates that the data storage location field points to a data structure, allocate at least one reference storage subspace for the data object to be written from multiple storage subspaces, wherein the bit corresponding to the reference storage subspace is not set; when writing the data object to at least one reference storage subspace, perform a setting operation on the bit corresponding to the non-reference storage subspace in the bitmap object.

[0135] Optionally, the above-mentioned data processing device also includes: a bitmap clearing unit, which is used to: when the current system load parameters meet the target conditions, determine the current storage unit to be processed from a plurality of storage units configured with respective matching bitmap objects; clear the storage space in the current storage unit that does not store valid data; clear the bitmap objects that do not match the current storage unit; and update the current unit description information of the current storage unit, wherein the updated current unit description information is used to indicate the storage status of the current storage unit.

[0136] According to another aspect of the embodiment of the present application, an electronic device for implementing the above data processing method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a mobile phone or a computer as an example. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0137] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0138] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0139] S1, in response to a data writing operation, obtaining unit description information of a storage unit allocated for a data object to be written;

[0140] S2, when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, creating a bitmap object that does not match the storage unit, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit;

[0141] S3, when writing the data object into at least one target storage subspace among the multiple storage subspaces, performing a set operation on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate the storage status of the at least one target storage subspace.

[0142] Alternatively, a person skilled in the art may understand that: Figure 6 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, etc.) shown in the figure, or with different Figure 5 Different configurations are shown.

[0143] Among them, the memory 602 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for adjusting the read voltage in the memory in the embodiment of the present application. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, the above-mentioned method for adjusting the read voltage in the memory is realized. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include a memory arranged relative to the processor 604 process, and these process memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 602 may be specifically, but not limited to, used to store information such as page elements and page styles. As an example, such as Figure 6 As shown, the memory 602 may include, but is not limited to, the information acquisition unit 502, the bitmap creation unit 504, and the setting unit 506 in the data processing device. In addition, it may also include, but is not limited to, other module units in the read voltage adjustment device in the memory, which will not be repeated in this example.

[0144] Optionally, the transmission device 606 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 606 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 606 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0145] In addition, the electronic device further includes: a display 608 for displaying the target page; and a connection bus 610 for connecting various module components in the electronic device.

[0146] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a point-to-point network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the point-to-point network.

[0147] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations;

[0148] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0149] S1, in response to a data writing operation, obtaining unit description information of a storage unit allocated for a data object to be written;

[0150] S2, when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, creating a bitmap object that does not match the storage unit, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit;

[0151] S3, when writing the data object into at least one target storage subspace among the multiple storage subspaces, performing a set operation on at least one target bit in the bitmap object, wherein the at least one target bit is respectively used to indicate the storage status of the at least one target storage subspace.

[0152] Optionally, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0153] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0154] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0155] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical functional division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] The above are only preferred implementations of the present application. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the protection scope of the present application.

[0160] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0161] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0162] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0163] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0164] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0165] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0166] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0167] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0168] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0169] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0170] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: include: In response to a data write operation, obtain unit description information of a storage unit allocated to a data object to be written; when the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, create a bitmap object matching the storage unit, wherein a plurality of bits in the bitmap object are respectively used to indicate the storage status of a plurality of storage subspaces in the storage unit; when writing the data object to at least one target storage subspace among the plurality of storage subspaces, perform a set operation on at least one target bit in the bitmap object, wherein at least one of the target bits is respectively used to indicate the storage status of at least one of the target storage subspaces.

2. The method according to claim 1, characterized in that After creating the bitmap object matching the storage unit, the method further includes: Obtaining a first storage location of a data structure for storing the bitmap object; updating the data storage location field in the unit description information according to the first storage location, wherein the data storage location field before updating is used to indicate the second storage location of the storage unit; The target identification field in the unit description information is updated, wherein the target identification field before the update is used to indicate that the data storage location field points to the storage unit, and the target identification field after the update is used to indicate that the data storage location field points to the data structure.

3. The method according to claim 2, characterized in that Before obtaining the storage location of the data structure for storing the bitmap object, the method further includes: Acquire the second storage location of the storage unit, and determine the second storage location as the first description information in the data structure; Determine the number of digits in the bitmap object as second description information in the data structure; The bitmap object is determined as the third description information in the data structure.

4. The method according to claim 2, characterized in that: After obtaining the unit description information of the storage unit allocated for the data object to be written, the method further includes: When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is equal to the first amount of data of the data object, writing the data object into the remaining storage space of the storage unit; When the unit description information indicates that the amount of reference data that can be stored in the remaining storage space of the storage unit is less than the first data amount of the data object, the first data portion of the data object is written into the remaining storage space of the storage unit, and the unit description information of the reference storage unit allocated to the second data portion of the data object is obtained.

5. The method according to claim 2, characterized in that: The creating a bitmap object matching the storage unit includes one of the following: Determine the target number as the number of bits included in the bitmap object, wherein the target number is the ratio between the total amount of data that can be stored in the storage unit and the minimum amount of modified data in the current file management system, and the amount of data that can be stored in the storage subspace is the minimum amount of modified data; Obtain the ratio between the total amount of data that can be stored in the storage unit and the first amount of data of the data object; determine the ratio between the total amount of data and the first amount of data as the number of bits included in the bitmap object, wherein the amount of data that can be stored in the storage subspace is the first amount of data.

6. The method according to any one of claims 1 to 5, characterized in that In the case where the data object is written into at least one target residual subspace among the plurality of residual subspaces, after performing a setting operation on at least one target bit in the bitmap object, the method further comprises: In response to a data read operation, obtaining unit description information of at least one of the storage units for storing a data object; and determining the data structure according to the data storage location field when a target identification field in the unit description information indicates that the data storage location field points to a data structure; Acquire a second storage location indicating the storage unit from the data structure, and acquire a reference bitmap object from the data structure; In a case where at least one reference bit in the reference bitmap object is set, at least one data object is obtained from a storage unit located at a second storage position according to the storage subspace corresponding to at least one of the bits.

7. The method according to any one of claims 1 to 5, characterized in that After performing the setting operation on at least one target bit in the bitmap object, the method further includes: In the case where the current system load parameter meets the target condition, determining a current storage unit to be processed from a plurality of storage units configured with the respective matching bitmap objects; Clearing the storage space in the current storage unit that does not store valid data; Clear the bitmap object matching the current storage unit; The current unit description information of the current storage unit is updated, wherein the updated current unit description information is used to indicate the storage state of the current storage unit.

8. A data processing device, characterized in that: include: An information acquisition unit, configured to acquire, in response to a data writing operation, unit description information of a storage unit allocated for a data object to be written; A bitmap creation unit, configured to create a bitmap object matching the storage unit when the unit description information indicates that the reference data amount that can be stored in the remaining storage space of the storage unit is greater than the first data amount of the data object, wherein a plurality of bits in the bitmap object are respectively used to indicate storage states of a plurality of storage subspaces in the storage unit; A setting unit is used to perform a setting operation on at least one target bit in the bitmap object when writing the data object to at least one target storage subspace among multiple storage subspaces, wherein at least one of the target bit is used to indicate the storage status of at least one of the target storage subspaces.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images