Data writing method and device and medium

Abstract

The invention provides a data writing method and device and a medium, and the method comprises the steps: responding to a writing request sent by a client, determining a target internal object identifier corresponding to the writing request and a target space of a target internal object from a server identifier distribution table, and marking that the target space is being used in the server identifier distribution table, the internal object is a part of storage space in the data pool; writing the data of the write request into the target space, and marking that the target space is used in a server identification distribution table; and when all the spaces of the target internal object identifier are marked as being used, clearing the target internal object identifier from the server identifier allocation table, and feeding back that the data is written to the client, so that the server identifier allocation table is introduced to manage the identifier information of the internal object when the data is written. And each write request can be correctly distributed to the corresponding storage space, so that the problem of data write-in conflict is avoided.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a data writing method, device, and medium. Background Art

[0002] In the scenario of data storage and management, the append write mode is widely used as an efficient writing method. It allows data to be continuously appended on the basis of the original data, avoiding the cumbersome process of rewriting the entire data set, thereby improving the efficiency and flexibility of data processing.

[0003] However, the data writing method in the related art does not introduce a mutual exclusion protection mechanism in the append write mode. This means that when multiple clients send write requests to the server simultaneously, the server may process these requests simultaneously, unable to ensure that each write request can be correctly allocated to the corresponding storage space, resulting in overlapping or chaos of the data writing positions. Summary of the Invention

[0004] The present disclosure provides a data writing method, device, and medium. By introducing a server identifier allocation table to manage the identifier information of internal objects during data writing, it ensures that each write request can be correctly allocated to the corresponding storage space, thereby avoiding the problem of data writing conflicts.

[0005] An embodiment of the first aspect of the present disclosure proposes a data writing method, which is applied to a server and includes: in response to a write request sent by a client, determining a target internal object identifier corresponding to the write request and a target space of the target internal object from the server identifier allocation table, and marking the target space as being in use in the server identifier allocation table, where the internal object is a part of the storage space in the data pool; writing the data of the write request into the target space, and marking the target space as having been used in the server identifier allocation table; when all spaces of the target internal object identifier are marked as having been used, clearing the target internal object identifier from the server identifier allocation table, and feeding back to the client that the data has been written.

[0006] In some embodiments of the present disclosure, determining a target internal object identifier corresponding to a write request and a target space of a target internal object from a server identifier allocation table includes: if the target internal object identifier does not exist in the server identifier allocation table, determining whether the target internal object identifier exists in an unallocated identifier management table; if the target internal object identifier exists in the unallocated identifier management table, obtaining the target internal object identifier from the unallocated identifier management table and storing the target internal object identifier in the server identifier allocation table; if the target internal object identifier does not exist in the unallocated identifier management table, generating a to-be-processed identifier and storing the to-be-processed identifier in a used identifier management table; performing an internal object creation operation on the to-be-processed identifier in the used identifier management table, using the to-be-processed identifier as the target internal object identifier, and storing the target internal object identifier in the unallocated identifier management table, so as to store the target internal object identifier in the server identifier allocation table by using the unallocated identifier management table.

[0007] In some embodiments of the present disclosure, before marking that the target space has been used up in the server identifier allocation table, the method includes: when data is completely written into the target space, storing location information of the data in a log, where the location information includes a logical location of the data and a physical location of the data in a data pool; storing log mapping information corresponding to the target internal object identifier in a log identifier association table, where the log mapping information is a mapping relationship between the target internal object identifier and a log number in the log.

[0008] In some embodiments of the present disclosure, the method includes: when the log meets the condition for a flush-down process, performing an aggregated flush-down process on the data stored in the log and the location information of the data to update the location information of the data stored in the log; when the aggregated flush-down process is completed, releasing the log mapping relationship of the data stored in the log, and determining whether the target internal object identifier corresponding to the log mapping relationship is stored in the server identifier allocation table and the client identifier allocation table to obtain a determination result; based on the determination result, clearing the target object identifier from the used identifier management table.

[0009] In some embodiments of the present disclosure, the method further includes: traversing all internal object identifiers existing in the data pool to determine whether the internal object identifiers in the data pool exist in the used identifier management table; if a first internal object identifier in the data pool does not exist in the used identifier management table, using the current time as the generation time of the first internal object identifier and storing the first internal object identifier in the client identifier allocation table and the used identifier management table.

[0010] In some embodiments of the present disclosure, the method further includes: receiving an identification application request sent by a client, determining an available internal object identification corresponding to the identification application request in an unallocated identification management table, and sending the available internal object identification to the client, so that the client writes the data of the write request into the available space of the available internal object identification.

[0011] In some embodiments of the present disclosure, the method further includes: when receiving a log write request sent by a client, storing the location information of the data of the log write request into the log, where the location information includes the logical location of the data and the physical location of the data in the data pool; storing the log mapping information corresponding to the available internal object identification in a log identification association table, where the log mapping information is a mapping relationship between the available internal object identification and the log number in the log.

[0012] An embodiment of the second aspect of the present disclosure provides a data writing method, which is applied to a client and includes: in response to a data write request, sending the write request to a server, so that the server writes the data of the write request into a target space based on a server identification allocation table in response to the write request; receiving the information that the data has been written sent by the server.

[0013] In some embodiments of the present disclosure, in response to a data write request, obtaining a to-be-processed internal object identification from a client identification manager; selecting an available internal object identification from the to-be-processed internal object identifications based on the generation time of the to-be-processed internal object identification, and determining the available space of the available internal object identification, so as to write the data of the write request into the available space; when the data of the write request is completely written into the available space, sending a log write request to the server, so that the server stores the location information of the data into the log based on the log write request, where the location information includes the logical location of the data and the physical location of the data in the data pool.

[0014] An embodiment of the third aspect of the present disclosure provides an electronic device, including: a processor and a memory for storing a computer program that can run on the processor, where the processor is configured to execute the method described in the embodiment of the first aspect of the present disclosure when running the computer program.

[0015] An embodiment of the fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute the method described in the embodiment of the first aspect of the present disclosure.

[0016] In summary, according to a data writing method provided by the present disclosure, it includes: in response to a write request sent by a client, determining a target internal object identifier corresponding to the write request and a target space of the target internal object from a server identifier allocation table, and marking the target space as being in use in the server identifier allocation table, where the internal object is a part of the storage space located in the data pool; writing the data of the write request into the target space, and marking the target space as having been used up in the server identifier allocation table; when all spaces of the target internal object identifier are marked as having been used up, clearing the target internal object identifier from the server identifier allocation table, and feeding back to the client that the data has been written, thereby realizing the introduction of the server identifier allocation table to manage the identifier information of internal objects during data writing, ensuring that each write request can be correctly allocated to the corresponding storage space, and thus avoiding the problem of data writing conflicts.

[0017] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present disclosure, and are used together with the description to explain the principles of the present disclosure, and do not constitute an improper limitation to the present disclosure.

[0019] Figure 1 It is a schematic flow chart of a data writing method provided by an embodiment of the present disclosure;

[0020] Figure 2 It is a schematic flow chart of another data writing method provided by an embodiment of the present disclosure;

[0021] Figure 3 It is a schematic flow chart of a data writing method provided by an embodiment of the present disclosure;

[0022] Figure 4 It is a schematic diagram of a data writing method provided by an embodiment of the present disclosure;

[0023] Figure 5 It is a schematic structural diagram of a data writing device provided by an embodiment of the present disclosure;

[0024] Figure 6 It is a schematic structural diagram of a data writing device provided by an embodiment of the present disclosure;

[0025] Figure 7 It is a schematic diagram of the hardware composition structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The following describes exemplary embodiments of the present disclosure in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0027] In the scenario of data storage and management, the append write mode is widely used as an efficient writing method. It allows data to be continuously appended based on the original data, avoiding the cumbersome process of rewriting the entire dataset, thereby improving the efficiency and flexibility of data processing.

[0028] However, the data writing method in the related art does not introduce a mutex protection mechanism in the append write mode. This means that when multiple clients simultaneously send write requests to the server, the server may process these requests simultaneously, unable to ensure that each write request can be correctly allocated to the corresponding storage space, resulting in overlapping or confusion of the data writing positions.

[0029] To solve the technical problems existing in the related art, embodiments of the present disclosure provide a data writing method.

[0030] The embodiments of the present disclosure will be described in detail below.

[0031] As Figure 1 shown, embodiments of the present disclosure provide a data writing method, which is applied to a server and includes the following steps:

[0032] Step 101, in response to a write request sent by a client, determine the target internal object identifier and the target space of the target internal object corresponding to the write request from the server identifier allocation table, and mark the target space as being used in the server identifier allocation table.

[0033] In the present disclosure, an internal object is a part of the storage space located in the data pool.

[0034] In some embodiments, when a write request sent by a client is received, if the data of the write request is strip-aligned data, at this time, it enters the server direct write processing process, that is, determine the target internal object identifier and the target space of the target internal object corresponding to the write request from the server identifier allocation table, and mark the target space as being used in the server identifier allocation table (that is, update the internal object space allocation information of the target internal object).

[0035] Among them, the striped-aligned data refers to the data whose length meets the striped alignment condition of the data pool. The target internal object refers to the internal object that can carry the write request, that is, the available space of the internal object can meet the requirement of the write data length of this write request.

[0036] It should be noted that when the data of the write request is not striped-aligned data, the data is written into the log for persistence.

[0037] Step 102, write the data of the write request into the target space, and mark the target space as having been used up in the server identification allocation table.

[0038] In some embodiments, after obtaining the available target internal object identifier and the corresponding target space (i.e., the write location information), write the data of the write request into the target space in the data pool.

[0039] It can be understood that the target space is part or all of the storage space that can be used in the target internal object.

[0040] After writing the data of the write request into the target space in the data pool, the present disclosure can further update the log association table

[0041] Step 103, when all the spaces of the target internal object identifier are marked as having been used up, remove the target internal object identifier from the server identification allocation table, and feedback to the client that the data has been written.

[0042] In some embodiments, when the update of the log association table in the present disclosure is completed, and all the spaces of the target internal object identifier are marked as having been used up, that is, it is determined that all the spaces of the target internal object identifier in the present disclosure have been used up, at this time, remove the target internal object identifier from the server identification allocation table, and feedback the information that the data has been written to the client, that is, reply to the client response.

[0043] In summary, according to the data writing method proposed by the present disclosure, it includes: in response to a write request sent by a client, determine the target internal object identifier and the target space of the target internal object corresponding to the write request from the server identification allocation table, and mark the target space as being used in the server identification allocation table, and the internal object is part of the storage space located in the data pool; write the data of the write request into the target space, and mark the target space as having been used up in the server identification allocation table; when all the spaces of the target internal object identifier are marked as having been used up, remove the target internal object identifier from the server identification allocation table, and feedback to the client that the data has been written, realizing the introduction of the server identification allocation table to manage the identification information of the internal object during data writing, ensuring that each write request can be correctly allocated to the corresponding storage space, thereby avoiding the problem of data writing conflict.

[0044] Figure 2 Further shown is a flowchart of a data writing method proposed by the present disclosure. Based on Figure 1 the embodiments shown, Figure 2 the following steps may be included.

[0045] Step 201, in response to a write request sent by a client, determine a target internal object identifier corresponding to the write request and a target space of the target internal object from a server identifier allocation table, and mark the target space as being in use in the server identifier allocation table.

[0046] In the present disclosure, an internal object is a partial storage space located in a data pool.

[0047] In some embodiments, if the target internal object identifier cannot be obtained from the server identifier allocation table, obtain a new available target internal object identifier from an unallocated identifier management table, and add the newly obtained target internal object identifier to the server identifier allocation table.

[0048] Specifically, if the target internal object identifier does not exist in the server identifier allocation table, determine whether the target internal object identifier exists in the unallocated identifier management table; if the target internal object identifier exists in the unallocated identifier management table, obtain the target internal object identifier from the unallocated identifier management table, and store the target internal object identifier in the server identifier allocation table.

[0049] If the target internal object identifier does not exist in the unallocated identifier management table, trigger an internal object pre-application process, generate a to-be-processed identifier, and store the to-be-processed identifier in a used identifier management table; then initiate an internal object creation transaction to perform an internal object creation operation on the to-be-processed identifier in the used identifier management table. After the internal object creation transaction is completed, use the to-be-processed identifier as the target internal object identifier, and store the target internal object identifier in the unallocated identifier management table to store the target internal object identifier in the server identifier allocation table using the unallocated identifier management table.

[0050] Wherein, when the internal object creation transaction is completed, if the transaction fails, clear the failed to-be-processed identifier from the used identifier management table.

[0051] Step 202, write the data of the write request into the target space.

[0052] Step 203, when the data is completely written into the target space, store the location information of the data in a log.

[0053] In the present disclosure, the location information includes the logical location of the data and the physical location of the data in the data pool.

[0054] In some embodiments, when the data is completely written to the target space, log writing is continued to be initiated to write the logical location of the data and the physical location of the data in the data pool into the log.

[0055] Step 204: Store the log mapping information corresponding to the target internal object identifier in the log identifier association table.

[0056] In the present disclosure, the log mapping information is a mapping relationship between a target internal object identifier and a log number in a log.

[0057] In some embodiments, after the log writing is completed, a mapping relationship between the target internal object identifier used in this write request and the corresponding log number is added to the log identifier association table.

[0058] Step 205: Mark the target space as finished in the server identification allocation table.

[0059] In some embodiments, after the log mapping information is written into the log identifier association table, the information in the server identifier allocation table is updated to mark that the target space has been used up.

[0060] Step 206: When all spaces of the target internal object identifier are marked as having been used up, the target internal object identifier is cleared from the server identifier allocation table, and a feedback is given to the client indicating that the data has been written.

[0061] In some embodiments, the server of the present disclosure can also implement an aggregated refresh process, that is, when the log meets the refresh processing conditions, the data stored in the log and the location information of the data are aggregated and refreshed to update the location information of the data stored in the log; when the aggregated refresh processing is completed, the log mapping relationship of the data stored in the log is released, and it is determined whether the target internal object identifier corresponding to the log mapping relationship is stored in the server identifier allocation table and the client identifier allocation table to obtain a judgment result; based on the judgment result, the target object identifier is cleared from the identifier management table in use.

[0062] Specifically, when the log writing meets certain conditions (for example, the number of non-stripe alignment in the log is equal to the preset number or the log writing time reaches the preset period), the aggregation and flushing process of the non-stripe alignment data is triggered, that is, the non-stripe alignment data write request recorded in the log, the location information of the stripe alignment data (i.e., metadata information), and the metadata update information of other write requests are aggregated and processed. After the non-stripe alignment data is aggregated, if there is data written in the flushing process, it is necessary to obtain the internal object identifier from the unallocated identifier management table, and use the internal object to carry the aggregated data of the non-stripe alignment data; if there is only the location information (metadata information) of the stripe alignment data, and there is no non-stripe alignment data process, then there is no data writing, and at this time, it is not necessary to obtain the internal object identifier from the unallocated identifier management table.

[0063] For the downbrush process with data written, the server of the present disclosure may initiate a data pool writing process (refer to the operations in steps 101 to 103); for the downbrush process without data written, the data pool writing process is not required. After the data pool writing is completed, the metadata information update process continues (for example, storing the location information of the aggregated data of non-strip-aligned data in the log, and storing the log mapping information of the internal object identifier corresponding to the aggregated data in the log identifier association table). After the metadata is updated, the server aggregation downbrush is completed.

[0064] After the server aggregation downbrush is completed, if the complete downbrush of a log object is completed in this downbrush, the mapping relationship between the log object and the internal object identifier is cleared from the log identifier management table. If the internal object identifier is completely removed from the log identifier management table, it is checked whether the internal object identifier is still in the server identifier allocation table. If not, it is continued to check whether the internal object identifier is still in the client identifier allocation table. If neither exists, the internal object identifier is removed from the in-use identifier management table.

[0065] The server of the present disclosure continues to traverse and check the internal object identifiers in the client identifier allocation table. If the generation time of the internal object identifier has not expired, no processing is required; if the generation time has expired, the corresponding internal object identifier will not be used by the client anymore, and it can be removed from the client identifier allocation table. After removal, it is continued to check whether the internal object identifier still has a mapping in the log identifier management table. If no mapping exists, the internal object identifier is removed from the in-use identifier management table, and then the clearing check process of the internal object is triggered, that is, when the available space amount is 0, the clearing is directly triggered, and when the available space amount is non-0, it enters the garbage collection management queue.

[0066] Furthermore, the server in the present disclosure may also implement a garbage collection process, that is, traverse all the internal object identifiers existing in the data pool to determine whether there are internal object identifiers in the data pool in the in-use identifier management table; if the first internal object identifier in the data pool does not exist in the in-use identifier management table, the current time is used as the generation time of the first internal object identifier, and the first internal object identifier is stored in the client identifier allocation table and the in-use identifier management table.

[0067] Specifically, the internal objects included in the identity management table during use may still be used, so garbage collection cannot be performed on these internal objects. In the scanning process of garbage collection in the present disclosure, the internal object information existing in the cluster will be scanned. When scanning, it is checked whether the internal object identifier scanned is already in the identity management table in use. If not, the internal object identifier is added to the client identity allocation table with the current time as the generation time, and the identifier is added to the identity management table in use. If it is already in the identity management table in use, no processing is performed. Among them, for the internal object information generated during the update process, before entering the garbage collection management queue, the present disclosure needs to check whether it is in the identity management table in use. If not, it is added to the garbage collection management queue. If it is, it does not enter the garbage collection management queue.

[0068] In addition, the server of the present disclosure can also perform auxiliary operations for client direct writing during the client direct writing process. That is, when the client cannot obtain available internal object information from the client identity manager, the client sends an identity application request for internal objects to the server. The server receives the identity application request sent by the client, determines the available internal object identifier corresponding to the identity application request in the unallocated identity management table, and sends the available internal object identifier to the client so that the client can write the data of the write request into the available space of the available internal object identifier.

[0069] Specifically, after the client direct writing identity service of the server receives the internal object identity application request from the client, it obtains the available internal object identifier from the unallocated identity management table, adds the available internal object identifier to the client identity allocation table of the server, and returns the available internal object identifier to the client.

[0070] After the client writes the data of the write request into the available space of the available internal object identifier, the client sends a log write request to the server. The server writes the logical position and the physical position of the data pool of the data into the log. The server log write processing process is the same as the processing process of client direct writing. That is, when the server receives the log write request sent by the client, it stores the position information of the data of the log write request in the log. The position information includes the logical position of the data and the physical position of the data in the data pool. The log mapping information corresponding to the available internal object identifier is stored in the log identifier association table. The log mapping information is the mapping relationship between the available internal object identifier and the log number in the log.

[0071] In summary, according to the data writing method proposed in the present disclosure, under the append write mechanism, by maintaining the unallocated identifier management table, the in-use identifier management table, the client identifier allocation table, the server identifier allocation table, the log identifier association table, and the client identifier management table, the internal object management compatibility processing of multiple processes such as client strip-aligned data writing, server strip-aligned data writing, server aggregation and flushing, and server garbage collection is realized, avoiding inconsistent use of internal objects. Through this allocation and protection mechanism, the aligned data writing process between the server and the client in the append write mode can run normally, and this writing process can improve the system processing performance and avoid the performance bottleneck caused by network bandwidth.

[0072] As Figure 3 shown, an embodiment of the present disclosure provides a data writing method, which is applied to a client and includes the following steps:

[0073] Step 301, in response to a data write request, send the write request to the server so that the server, in response to the write request, writes the data of the write request into the target space based on the server identifier allocation table.

[0074] In some embodiments, in response to a data write request, the client may forward the write request to the server so that the server performs the direct write operation of the data in steps 201 to 206 above, which will not be elaborated here.

[0075] In addition, the client may also perform a client direct write operation according to the data write request. The client direct write further reduces the network bandwidth occupancy in the large bandwidth scenario on the basis of the server direct write mechanism. The client directly forwards the data to be written to the data pool for writing, avoiding the forwarding process of the server direct write.

[0076] That is, in response to a data write request, the client obtains the internal object identifier to be processed from the client identifier manager; based on the generation time of the internal object identifier to be processed, selects the available internal object identifier from the internal object identifiers to be processed, determines the available space of the available internal object identifier, and writes the data of the write request into the available space; when the data of the write request is completely written into the available space, send a log write request to the server so that the server, based on the log write request, stores the location information of the data in the log, and the location information includes the logical location of the data and the physical location of the data in the data pool.

[0077] Specifically, after the client receives a write request, if the data length of the write request meets the data pool strip alignment condition, that is, when the data is strip-aligned data, it enters the client direct write processing process. When the client performs data direct write processing, it first obtains the internal object identifier to be processed from the client representation manager. Among them, when the client identifier manager retrieves the internal object identifier to be processed, it sequentially traverses the saved internal object identifier information to be processed. When the generation time of the corresponding internal object identifier to be processed has expired, it is directly removed from the client identifier manager. If it has not expired, it is determined whether the remaining space of the internal object to be processed meets the requirements of the data of this write request. If it can be met, the retrieval is completed, the internal object identifier to be processed is removed from the manager, and the internal object identifier to be processed is returned to the client processing process as an available internal object identifier. If it cannot be met, it is skipped and the next one is continued to be checked.

[0078] When the client cannot obtain available internal object information from the client identifier manager, the client sends an internal object identifier application request to the server, so that after the client direct write identifier service of the server receives the internal object identifier application request of the client, it obtains an available internal object identifier from the unallocated identifier management table, adds the available internal object identifier to the client identifier allocation table of the server, and returns the available internal object identifier to the client.

[0079] After the client receives the available internal object identifier returned by the client direct write identifier service, it records the generation time of the available internal object identifier and generates available internal object identifier information; after the client obtains the available internal object identifier from the client identifier manager or the client direct write identifier service, it determines the available space corresponding to the available internal object identifier and sends a data write request to the data pool to write the data into the available space.

[0080] After the client completes writing data to the data pool, it sends a log write request to the server, and the server writes the logical position and physical position of the data pool of the data into the log. The server log write processing process is the same as the server direct write processing process, which will not be elaborated here.

[0081] After the client completes data writing, the available internal object identifiers with remaining available space will be added to the client identifier management table again. When adding, the expiration time will be checked. If it has not expired, it can be directly added. If it has expired, it will be directly discarded.

[0082] Step 302, receive the data written information sent by the server.

[0083] In summary, the data writing method proposed according to the present disclosure includes: in response to a write request sent by a client, determining a target internal object identifier corresponding to the write request and a target space of the target internal object from a server identifier allocation table, and marking the target space as being in use in the server identifier allocation table, where the internal object is a partial storage space located in a data pool; writing the data of the write request into the target space, and marking the target space as having been used in the server identifier allocation table; when all spaces of the target internal object identifier are marked as having been used, clearing the target internal object identifier from the server identifier allocation table, and feeding back to the client that the data has been written, thereby realizing the introduction of the server identifier allocation table to manage the identifier information of internal objects during data writing, ensuring that each write request can be correctly allocated to the corresponding storage space, and thus avoiding the problem of data writing conflicts.

[0084] Based on Figures 1 to 3 the embodiments shown, as Figure 4 shown, the present disclosure provides a schematic diagram of a specific data writing method.

[0085] Referring to Figure 4 the data writing process of the present disclosure includes an unallocated ID management table (i.e., the unallocated identifier management table in the present disclosure), a used ID management table (i.e., the used identifier management table in the present disclosure), a client ID allocation table (i.e., the client identifier allocation table in the present disclosure), a server ID allocation table (i.e., the server identifier allocation table in the present disclosure), a log ID association table (i.e., the log identifier association table in the present disclosure), and a client ID management table (i.e., the client identifier management table in the present disclosure).

[0086] In some embodiments, when an ID (i.e., the internal object identifier in the present disclosure) is applied for creation, it is added to the used ID management table. After successful creation, the available ID is added to the unallocated ID management table; for direct writing by the server and direct writing by the client, when an ID is needed during the process, the ID is obtained from the unallocated ID management table and added to the corresponding client ID allocation table and server ID allocation table; the server and the client write the data to the corresponding data pool location according to the obtained ID, then submit the log writing, and after the log writing is completed, record the association information between the log and the ID in the log ID association table, and remove the corresponding usage information from the server ID allocation table; after the log is successfully flushed, the corresponding log information can be released, and at this time, remove the log association information (i.e., the log mapping information in the present disclosure) from the log association table, and remove the expired information from the client ID allocation table; when the ID is removed from the client ID allocation table, the log ID association table, and the server ID allocation table, it can be removed from the used ID management table; the information in the client ID allocation table and the client ID management table is managed according to a limited available period, and an expiration check is triggered when log deletion, ID acquisition, and insertion events occur.

[0087] To implement the data writing method provided by the embodiments of the present disclosure, the embodiments of the present disclosure further provide a data writing device. As Figure 5 shown, the data writing device 500 is applied to the server side and includes:

[0088] A determination unit 510, configured to, in response to a write request sent by a client, determine a target internal object identifier corresponding to the write request and a target space of the target internal object from a server identifier allocation table, and mark the target space as being in use in the server identifier allocation table. The internal object is a part of the storage space located in the data pool;

[0089] A writing unit 520, configured to write the data of the write request into the target space, and mark the target space as having been used up in the server identifier allocation table;

[0090] A feedback unit 530, configured to, when all spaces of the target internal object identifier are marked as having been used up, clear the target internal object identifier from the server identifier allocation table, and feedback to the client that the data has been written.

[0091] In some embodiments, the determination unit 510 is configured to: if the target internal object identifier does not exist in the server identifier allocation table, determine whether the target internal object identifier exists in the unallocated identifier management table; if the target internal object identifier exists in the unallocated identifier management table, obtain the target internal object identifier from the unallocated identifier management table, and store the target internal object identifier in the server identifier allocation table; if the target internal object identifier does not exist in the unallocated identifier management table, generate a to-be-processed identifier, and store the to-be-processed identifier in the in-use identifier management table; perform an internal object creation operation on the to-be-processed identifier in the in-use identifier management table, use the to-be-processed identifier as the target internal object identifier, and store the target internal object identifier in the unallocated identifier management table, so as to store the target internal object identifier in the server identifier allocation table by using the unallocated identifier management table.

[0092] In some embodiments, the device further includes: a log storage unit, configured to, before marking the target space as having been used up in the server identifier allocation table, when the data is completely written into the target space, store the location information of the data in the log. The location information includes the logical location of the data and the physical location of the data in the data pool; store the log mapping information corresponding to the target internal object identifier in the log identifier association table. The log mapping information is a mapping relationship between the target internal object identifier and the log number in the log.

[0093] In some embodiments, the apparatus further includes: a downbrush processing unit, configured to perform an aggregated downbrush process on the data stored in the log and the location information of the data when the log meets the downbrush processing condition, so as to update the location information of the data stored in the log; when the aggregated downbrush process is completed, release the log mapping relationship of the data stored in the log, and determine whether the target internal object identifier corresponding to the log mapping relationship is stored in the server identifier allocation table and the client identifier allocation table, to obtain a determination result; based on the determination result, clear the target object identifier from the in-use identifier management table.

[0094] In some embodiments, the apparatus further includes: a recycling unit, configured to traverse all internal object identifiers existing in the data pool, and determine whether there is an internal object identifier in the data pool in the in-use identifier management table; if there is no first internal object identifier in the data pool in the in-use identifier management table, use the current time as the generation time of the first internal object identifier, and store the first internal object identifier in the client identifier allocation table and the in-use identifier management table.

[0095] In some embodiments of the present disclosure, the feedback unit 530 is further configured to: receive an identifier application request sent by a client, determine an available internal object identifier corresponding to the identifier application request in the unallocated identifier management table, and send the available internal object identifier to the client, so that the client writes the data of the write request into the available space of the available internal object identifier.

[0096] In some embodiments of the present disclosure, the feedback unit 530 is further configured to: when receiving a log write request sent by a client, store the location information of the data of the log write request in the log, where the location information includes the logical location of the data and the physical location of the data in the data pool; store the log mapping information corresponding to the available internal object identifier in the log identifier association table, where the log mapping information is the mapping relationship between the available internal object identifier and the log number in the log.

[0097] To implement the data writing method provided by the embodiments of the present disclosure, the embodiments of the present disclosure further provide a data writing apparatus, as Figure 6 shown, the data writing apparatus 600, which is applied to a client and includes:

[0098] A forwarding unit 610, configured to send a write request to a server in response to a data write request, so that the server writes the data of the write request into a target space based on the server identifier allocation table in response to the write request;

[0099] A receiving unit 620, configured to receive the data written information sent by the server.

[0100] In some embodiments, the apparatus further includes: a client direct write unit, configured to, in response to a data write request, obtain a to-be-processed internal object identifier from a client identifier manager; select an available internal object identifier from the to-be-processed internal object identifiers based on the generation time of the to-be-processed internal object identifier, and determine an available space of the available internal object identifier, so as to write the data of the write request into the available space; when the data of the write request is completely written into the available space, send a log write request to the server, so that the server stores the location information of the data into the log based on the log write request, where the location information includes the logical location of the data and the physical location of the data in the data pool.

[0101] It should be noted that: when the data writing apparatus provided in the above embodiment performs data writing, only the division of the above program modules is used for illustration. In practical applications, the above processing can be allocated to different program modules according to needs, that is, the internal structure of the data writing apparatus is divided into different program modules to complete all or part of the processing described above. In addition, the data writing apparatus provided in the above embodiment and the data writing method embodiment provided in the embodiments of the present disclosure belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be elaborated here.

[0102] Figure 7 It is a schematic diagram of the hardware composition structure of the electronic device provided in the embodiments of the present disclosure. As Figure 7 shown, the electronic device 700 includes at least one processor 702; and a memory 701 communicatively connected to the at least one processor 702; wherein, the memory 701 stores instructions executable by the at least one processor 702, and the instructions are executed by the at least one processor 702 to implement the steps of the data writing method provided in the embodiments of the present disclosure.

[0103] Optionally, the electronic device may specifically be the data writing apparatus in the embodiments of the present application, and the electronic device can implement the corresponding processes implemented by the data writing apparatus in each method of the embodiments of the present application. For the sake of brevity, it will not be elaborated here.

[0104] It can be understood that the electronic device further includes a communication interface 703. Each component in the electronic device is coupled together through a bus system 704. It can be understood that the bus system 704 is used to realize the connection and communication between these components. The bus system 704 includes not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 7 all kinds of buses are labeled as the bus system 704.

[0105] It can be understood that the memory 701 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read□Only Memory), an erasable programmable read-only memory (EPROM, Erasable Programmable Read□Only Memory), an electrically erasable programmable read-only memory (EEPROM, Electrically Erasable Programmable Read□Only Memory), a ferromagnetic random access memory (FRAM, ferromagnetic random access memory), a flash memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD□ROM, Compact Disc Read□Only Memory); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM, Random AccessMemory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory), dynamic random access memory (DRAM, Dynamic Random Access Memory), synchronous dynamic random access memory (SDRAM, SynchronousDynamic Random Access Memory), double data rate synchronous dynamic random access memory (DDRSDRAM, Double Data Rate Synchronous Dynamic Random Access Memory), enhanced synchronous dynamic random access memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), sync link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), direct rambus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 701 described in the embodiments of the present invention is intended to include but not limited to these and any other suitable types of memories.

[0106] The methods disclosed in the above embodiments of the present disclosure can be applied to or implemented by the processor 702. The processor 702 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above methods can be completed by the integrated logic circuit in the hardware of the processor 702 or the instructions in the form of software. The above-mentioned processor 702 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 702 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. Combining the steps of the methods disclosed in the embodiments of the present invention can be directly embodied as being executed by the hardware decoding processor, or executed by the combination of the hardware and software modules in the decoding processor. The software module may be located in the storage medium, and this storage medium is located in the memory 701. The processor 702 reads the information in the memory 701 and combines its hardware to complete the steps of the foregoing methods.

[0107] In an exemplary embodiment, the electronic device can be implemented by one or more application-specific integrated circuits (ASICs, Application Specific Integrated Circuit), DSPs, programmable logic devices (PLDs, Programmable Logic Device), complex programmable logic devices (CPLDs, Complex Programmable Logic Device), FPGAs, general-purpose processors, controllers, MCUs, microprocessors (Microprocessor), or other electronic components, and is used to execute the foregoing methods.

[0108] The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to cause the computer to execute the steps of the data writing method described in the embodiments of the present disclosure when executed.

[0109] The embodiments of the present disclosure also provide a computer program product, including a computer program, and the computer program implements the steps of the data writing method described in the embodiments of the present disclosure when executed by a processor.

[0110] Optionally, the computer-readable storage medium can be applied to the data writing device in the embodiments of the present application, and the computer instructions cause the computer to execute the corresponding processes implemented by the data writing device in each method of the embodiments of the present application. For the sake of brevity, it will not be elaborated here.

[0111] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be electrical, mechanical, or other forms.

[0112] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, each functional unit in the embodiments of the present invention can be all integrated in a processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in a unit; the above-mentioned integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0114] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as removable storage devices, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0115] Alternatively, if the above-mentioned integrated units of the present invention are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the foregoing storage medium includes: various media such as removable storage devices, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0116] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims described above.

Claims

1. A data writing method, characterized in that, The method is applied to the server, and the method includes: In response to a write request sent by the client, determine the target internal object identifier and the target space of the target internal object corresponding to the write request from the server identifier allocation table, and mark the target space in the server identifier allocation table as being in use, and the internal object is a portion of the storage space located in the data pool; Writing the data of the write request into the target space, and marking the target space as finished in the server identification allocation table; When all spaces of the target internal object identifier are marked as having been used up, the target internal object identifier is cleared from the server identifier allocation table, and feedback is given to the client indicating that the data has been written.

2. The method according to claim 1, wherein The step of determining the target internal object identifier and the target space of the target internal object corresponding to the write request from the server identifier allocation table includes: If the target internal object identifier does not exist in the server identifier allocation table, determining whether the target internal object identifier exists in the unallocated identifier management table; If the target internal object identifier exists in the unassigned identifier management table, obtaining the target internal object identifier from the unassigned identifier management table, and storing the target internal object identifier in the server identifier allocation table; If the target internal object identifier does not exist in the unassigned identifier management table, generating a pending identifier and storing the pending identifier in the in-use identifier management table; An internal object creation operation is performed on the pending identifier in the in-use identifier management table, the pending identifier is used as the target internal object identifier, and the target internal object identifier is stored in the unallocated identifier management table, so as to use the unallocated identifier management table to store the target internal object identifier in the server-side identifier allocation table.

3. The method according to claim 1, characterized in that, Before marking the target space as completed in the server identifier allocation table, the method includes: When the data is completely written to the target space, the location information of the data is stored in a log, the location information including the logical location of the data and the physical location of the data in the data pool; The log mapping information corresponding to the target internal object identifier is stored in the log identifier association table, where the log mapping information is a mapping relationship between the target internal object identifier and the log number in the log.

4. The method according to claim 3, wherein The method comprises: When the log meets the refresh processing condition, aggregate and refresh the data and the location information of the data stored in the log to update the location information of the data stored in the log; When the aggregated flushing process is completed, the log mapping relationship of the data stored in the log is released, and it is determined whether the target internal object identifier corresponding to the log mapping relationship is stored in the server identifier allocation table and the client identifier allocation table to obtain a determination result; Based on the determination result, the target object identifier is cleared from the in-use identifier management table.

5. The method according to claim 1, wherein The method further comprises: Traverse all internal object identifiers existing in the data pool to determine whether the internal object identifiers in the data pool exist in the in-use identifier management table; If the first internal object identifier in the data pool does not exist in the in-use identifier management table, use the current time as the generation time of the first internal object identifier, and store the first internal object identifier in the client identifier allocation table and the in-use identifier management table.

6. The method according to claim 1, characterized in that, The method further includes: Receive an identifier application request sent by the client, determine the available internal object identifier corresponding to the identifier application request in the unallocated identifier management table, and send the available internal object identifier to the client, so that the client writes the data of the write request into the available space of the available internal object identifier.

7. The method according to claim 6, characterized in that, The method further includes: When receiving the log write request sent by the client, store the location information of the data of the log write request in the log, where the location information includes the logical location of the data and the physical location of the data in the data pool; Store the log mapping information corresponding to the available internal object identifier in the log identifier association table, where the log mapping information is the mapping relationship between the available internal object identifier and the log number in the log.

8. A data writing method, characterized in that, The method is applied to the client, and the method includes: In response to a data write request, send the write request to the server, so that the server, in response to the write request, writes the data of the write request into the target space based on the server identifier allocation table; Receive the data written information sent by the server.

9. The method according to claim 8, wherein The method includes: In response to the data write request, obtain a to-be-processed internal object identifier from the client identifier manager; Based on the generation time of the to-be-processed internal object identifier, select the available internal object identifier from the to-be-processed internal object identifiers, and determine the available space of the available internal object identifier, so as to write the data of the write request into the available space; When the data of the write request is completely written into the available space, send a log write request to the server, so that the server, based on the log write request, stores the location information of the data in the log, where the location information includes the logical location of the data and the physical location of the data in the data pool.

10. An electronic device, characterized in that, Includes: A processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor is used to run the computer program, it executes the data writing method according to any one of claims 1-7 or 8-9.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instruction is used to make the computer execute the data writing method according to any one of claims 1-7 or 8-9.

Images