Data processing method, computing device, storage medium, and computer program product

Abstract

This specification relates to the field of database technology, and particularly to data processing methods, computing devices, storage media, and computer program products. The data processing method includes: in response to a data write request for a target data structure, determining the data to be written carried in the data write request; if the data to be written contains first data, allocating a first data storage node outside the target data structure to the first data when the target data structure is not locked, wherein the amount of the first data is greater than a preset data amount threshold; and writing the first data to the first data storage node when the target data structure is not locked, wherein the first data storage node is associated with the target data structure.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of database technology, and in particular to data processing methods, computing devices, storage media, and computer program products. Background Technology

[0002] In practical applications, large field types are widely used in the database field. They are typically used to store large amounts of data, such as log text, images, and binary streams. However, because large field data occupies a large amount of space, writing large field data to the database places high demands on database performance, and write performance is relatively low. Therefore, an effective technical solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising: In response to a data write request for a target data structure, determine the data to be written carried in the data write request; If it is determined that the data to be written contains first data, and the target data structure does not have a lock, a first data storage node outside the target data structure is allocated to the first data, wherein the amount of the first data is greater than a preset data amount threshold. If no lock exists in the target data structure, the first data is written to the first data storage node, wherein the first data storage node is associated with the target data structure.

[0005] According to a second aspect of the embodiments of this specification, a data processing apparatus is provided, comprising: The determination module is configured to determine the data to be written carried in the data write request in response to a data write request for a target data structure. The allocation module is configured to allocate a first data storage node outside the target data structure to the first data when it is determined that the data to be written contains first data, and the target data structure does not have a lock, wherein the amount of the first data is greater than a preset data amount threshold. The write module is configured to write the first data to the first data storage node when there is no lock on the target data structure, wherein the first data storage node is associated with the target data structure.

[0006] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the data processing method described above.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0009] One embodiment of this specification provides a data processing method, comprising: in response to a data write request for a target data structure, determining data to be written carried in the data write request; if it is determined that the data to be written includes first data, allocating a first data storage node outside the target data structure to the first data when the target data structure is not locked, wherein the data volume of the first data is greater than a preset data volume threshold; and writing the first data to the first data storage node when the target data structure is not locked, wherein the first data storage node is associated with the target data structure.

[0010] In the above method, after responding to a data write request for the target data structure, the data to be written carried by the data write request can be determined. If the data to be written contains the first data, since the data volume of the first data is greater than the preset data volume threshold, that is, the first data is a large field data, the first data cannot be written into the target data structure. In the absence of a lock on the target data structure, a first data storage node outside the target data structure can be allocated for the first data. And in the absence of a lock on the target data structure, the first data is written to the external first data storage node. The writing of the first data is performed without locking the first data storage node and without locking the target data structure, thus ensuring the writing performance of the first data. Attached Figure Description

[0011] Figure 1 This is a schematic diagram illustrating an application scenario of a data processing method provided in one embodiment of this specification; Figure 2This is a flowchart illustrating a data processing method provided in one embodiment of this specification; Figure 3 This is a schematic diagram of the target data structure in a data processing method provided in one embodiment of this specification; Figure 4 This is a schematic diagram illustrating the allocation of a first data storage node in a data processing method provided in one embodiment of this specification; Figure 5 This is a schematic diagram illustrating the writing of non-large field data in a data processing method provided in one embodiment of this specification; Figure 6 This is a schematic diagram illustrating the modification of external page pointers in a data processing method provided in one embodiment of this specification; Figure 7 This is a schematic diagram illustrating data deletion in a data processing method provided in one embodiment of this specification; Figure 8 This is a flowchart illustrating the processing procedure of a data processing method provided in one embodiment of this specification. Figure 9 This is a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification; Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0012] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0015] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0016] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0017] Large data fields: Data storage fields that occupy a large amount of space in the database, including both character objects and binary objects. Examples include images, audio files, video files, documents, and other types of multimedia content.

[0018] Mutex lock: Also known as a write lock, if a write lock exists on a leaf node, that leaf node cannot be accessed. It is used to prevent multiple threads from accessing shared resources at the same time. Other threads need to wait for the current thread to release the write lock before they can access it.

[0019] Shared lock: also known as read lock, is a synchronization mechanism used to solve concurrent access control problems, allowing multiple threads holding read locks to access shared resources simultaneously.

[0020] B-tree: A self-balancing tree-like data structure where all leaf nodes are at the same level, and each node can have multiple child nodes. It is a widely used data structure in database management systems to improve the speed of data retrieval.

[0021] Primary key index: A special type of index in a database, created based on the primary key of a table, typically using a B-tree index. A primary key is one or a set of fields used to uniquely identify each row in a table.

[0022] Page: The smallest unit of data organization in a database.

[0023] In practical applications, large field types are widely used in databases. Users can use large fields to store large amounts of data, such as log text, images, and binary streams, achieving persistence by writing them to the database. Concurrent writes of large fields pose a significant challenge to database performance because large fields themselves occupy a large amount of space and cannot be directly stored in fixed-size pages of the index. Furthermore, the long write time consumes more I / O bandwidth for log and data writes. During multi-threaded concurrent writes, it is usually necessary to hold mutex locks on the index and outer pages. Long-running operations on large fields consume even more lock holding time, reducing concurrent write performance. Even with higher I / O bandwidth, user write performance cannot be improved. Therefore, an effective technical solution is urgently needed to address these issues.

[0024] This specification provides a data processing method, and also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0025] See Figure 1 , Figure 1 The illustration shows an application scenario of a data processing method provided according to an embodiment of this specification, which specifically includes the following steps.

[0026] In response to a data write request for a target data structure, determine the data to be written carried in the data write request; If it is determined that the data to be written contains first data, and the target data structure does not have a lock, a first data storage node outside the target data structure is allocated to the first data, wherein the amount of the first data is greater than a preset data amount threshold. If no lock exists in the target data structure, the first data is written to the first data storage node, wherein the first data storage node is associated with the target data structure.

[0027] Specifically, Figure 1 It includes an end-side device 102 and a database 104, which uses a target data structure for data indexing and data storage.

[0028] In specific implementation, the user can send a data write request to the database 104 through the terminal device 102. The database 104 can determine the data to be written carried in the data write request. If it is determined that the data to be written contains first data with a data volume greater than a preset data volume threshold, the database 104 allocates a first data storage node outside the target data structure for the first data if the target data structure does not hold any lock. If the target data structure does not hold any lock, the first data is written to the first data storage node outside the target data structure, thereby realizing the writing of the first data.

[0029] The edge device 102 may include a browser, an app (application), or a web application such as an H5 (Hypertext Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. The edge device can be developed based on a software development kit (SDK) provided by the server, such as a real-time communication (RTC) SDK. The edge device can be deployed in an electronic device and depends on the device's operation or certain apps within the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured in the electronic device, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0030] See Figure 2 , Figure 2 A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0031] Step 202: In response to a data write request for a target data structure, determine the data to be written carried in the data write request.

[0032] The target data structure can be understood as the data structure used for data indexing and data storage in the database. In practical applications, it can be a primary key index, which can be understood as an index built on the primary key in the database. In the case of a relational database, the primary key index can be a B-tree structure. The data write request can be understood as a data write request to the database corresponding to the target data structure. The data to be written can be understood as the data that needs to be written to the database.

[0033] Specifically, the database can respond to a data write request sent by a user through a client for a target data structure and determine the data that needs to be written to the database, carried in the data write request.

[0034] Step 204: If it is determined that the data to be written contains the first data, and the target data structure does not have a lock, allocate a first data storage node outside the target data structure to the first data, wherein the amount of the first data is greater than a preset data amount threshold.

[0035] The target data structure does not have locks, which can be understood as the target data structure not having global read locks and write locks. The first data can be understood as data whose volume is greater than a preset data volume threshold. The first data can be data of the first data type, such as large field data. The first data storage node can be understood as an external page outside the target data structure, used to store the first data in the database.

[0036] Specifically, if it is determined that the data to be written contains large field data, an external page outside the target data structure can be allocated for the large field data, provided that the target data structure does not hold a global read lock or write lock.

[0037] In practical applications, see Figure 3 , Figure 3 This diagram illustrates a target data structure in a data processing method according to an embodiment of this specification, such as... Figure 3 As shown, the target data structure can be a B-tree (i.e., a primary key index). This target data structure includes multiple leaf nodes, each of which can store data. Each leaf node can contain row records and column data. Row records can store data index information, and column data can store the data itself. Outside this target data structure is an outer page list, consisting of multiple outer pages. These outer pages are used to store large field data. Specifically, in a relational database organized using a B-tree index, each leaf node of the B-tree stores a fixed amount of data. If a row of data cannot fit into a single leaf node, large field data, which occupies a significant amount of space, will be stored separately on a data page outside the primary key index. Therefore, the large field data is separated from the primary key index stored in the B-tree; the large field data is stored separately on an outer page outside the primary key index structure, and a reference to this outer page is stored in the primary key index.

[0038] In specific implementation, allocating a first data storage node outside the target data structure to the first data includes: Based on the amount of the first data, a first data storage node other than the target data structure is allocated to the first data, wherein the storage space of the first data storage node is greater than or equal to the amount of data.

[0039] Specifically, the number of external pages that need to be allocated can be calculated based on the amount of data in the first data, and external pages outside the target data structure can be allocated to the first data.

[0040] In practical applications, see Figure 4 , Figure 4 This diagram illustrates the allocation of a first data storage node in a data processing method according to an embodiment of this specification. Without holding any locks on the target data structure, the required number of external pages (i.e., the first data storage node) for each large field column (i.e., the amount of first data) in the written row record is calculated, and disk space for the external pages is allocated for the first data as the first data storage node. Further, to improve space allocation efficiency, an asynchronous allocation thread can be used to allocate the first data storage node. That is, the required external pages are allocated using an asynchronous allocation thread, and the thread writing the large field data in the foreground can allocate the required disk space for the external pages in batches from the file system while holding a file lock. For each large field data, the allocated external pages are chained together into a page linked list, and the content of each large field data is copied one by one to the external page linked list, thereby writing the content of the large field data that needs to be stored on the external pages.

[0041] In summary, since a global file lock is held only when allocating disk space from the file system, most time-consuming operations on large fields of data are lock-free, such as page initialization and data copying. This supports parallel write operations on multiple large fields of data.

[0042] Furthermore, after writing the first data to the first data storage node, the method further includes: Determine the reference information corresponding to the first data storage node; Based on the reference information, the first data storage node and the target data structure are associated.

[0043] The reference information of the first data storage node can be understood as information used to index from the target data structure to the first data storage node. For example, it can be the location information of the first data storage node, or it can be the link information that can be linked to the first data storage node.

[0044] Specifically, the location information of the first data storage node can be determined, and this location information can be used to associate the primary key index information of the first data storage node and the target data structure, so as to query the data content of the first data storage node through the primary key index.

[0045] In summary, by utilizing location information to manage the primary key index information of the first data storage node and the target data structure, the location of the first data storage node can be determined through the primary key index, and the data content in the first data storage node can be further queried.

[0046] In specific implementation, associating the first data storage node and the target data structure based on the reference information includes: The reference information is stored in the target data structure, and the association between the first data storage node and the target data structure is realized based on the reference information stored in the target data structure.

[0047] Specifically, the location information of the first data storage node can be stored in the primary key index to realize the association between the first data storage node and the target data structure.

[0048] Step 206: If there is no lock on the target data structure, write the first data to the first data storage node, wherein the first data storage node is associated with the target data structure.

[0049] The target data structure does not have a lock, which can be understood as the target data structure not having a read lock or a write lock.

[0050] Furthermore, after determining that the data to be written contains the first data, the method further includes: If it is determined that the data to be written also contains second data, the second data storage node corresponding to the second data is determined in the target data structure according to the data index information of the second data, wherein the data volume of the second data is less than the preset data volume threshold; Write the second data to the second data storage node.

[0051] Here, the second data can be understood as data with a volume smaller than a preset data volume threshold. The second data can be data of a second data type; that is, the first data type and the second data type are different. Therefore, the second data can be, for example, non-large field data. The second data storage node can be understood as a leaf node in the target data structure. Therefore, the second data storage node can include column data in the row records whose length is smaller than the preset data volume threshold.

[0052] Specifically, if it is determined that the data to be written includes first data and second data, the first data is allocated to the first data storage node as described above. For the second data, the second data storage node in the target data structure to be determined can be determined based on the data index information of the second data and the primary key recorded in the row record included in the second data storage node, and the second data is written to the second data storage node.

[0053] In summary, when it is determined that the data to be written contains both large and non-large fields, different processing methods can be applied to the large and non-large fields to achieve the writing of data for both fields.

[0054] In specific implementation, determining the second data storage node corresponding to the second data in the target data structure based on the data index information of the second data includes: If a read lock exists in the target data structure, the second data storage node corresponding to the second data is determined in the target data structure based on the data index information of the second data. The step of writing the second data to the second data storage node includes: If a write lock exists on the second data storage node, the second data is written to the second data storage node.

[0055] The data index information of the second data can be used to determine which leaf node in the target data structure the second data is stored in. The read lock on the target data structure can be understood as a global read lock on the target data structure.

[0056] Specifically, if the target data structure holds a global read lock, the second data storage node where the second data is about to be written can be determined in the target data structure based on the data index information of the second data, and a write lock can be added to the second data storage node. If the second data storage node holds a write lock, the second data can be written to the second data storage node.

[0057] In practical applications, see Figure 5 , Figure 5 This diagram illustrates the writing of non-large field data in a data processing method according to an embodiment of this specification. Figure 5As shown, when the target data structure holds a global read lock, starting from the root node of the target data structure, the process searches for the leaf node (i.e., the second data storage node) where the data to be written is about to be written. Then, a write lock is acquired on this second data storage node. First, the column data of non-large fields stored in the primary key index is modified, while reserving space for the column data of large fields. Through the transaction locking system's marking of this row record, other threads cannot modify the current row record. Based on this, the global read lock on the target data structure and the write lock on the second data storage node can be released. Since only the write lock on the second data storage node exists at this stage, and the global write lock on the target data structure is not held, parallel write operations on multiple large fields can be supported.

[0058] Furthermore, after writing the second data to the second data storage node when a write lock exists on the second data storage node, the method further includes: Release the read lock on the target data structure and the write lock on the second data storage node.

[0059] Specifically, after writing non-large field data to the second data storage node, the read lock of the target data structure and the write lock of the second data storage node can be released, which facilitates subsequent processing of the data in the target data structure and the second data storage node.

[0060] In practical applications, storing the reference information into the target data structure includes: If a read lock exists in the target data structure, the reference information is stored in the target data structure.

[0061] Specifically, when the target data structure holds a read lock and the second data storage node holds a write lock, the reference information of the first data storage node is stored in the target data structure.

[0062] In practical applications, see Figure 6 , Figure 6 This diagram illustrates an external page pointer modification in a data processing method according to an embodiment of this specification, where the data to be written includes both large-field data and non-large-field data, in conjunction with the above... Figure 5It is known that before the content of the large field data is written to the external page, the row records in the target data structure only modify the content of the primary key index, reserving the space in each large field data pointing to the external page, without actually writing it to the actual external page location. Based on this, with the target data structure holding a global read lock on the primary key index and a write lock on the corresponding leaf node (i.e., the second data storage node), the location of the external page linked list corresponding to the already written large field data (i.e., the reference information of the first data storage node) is stored in the primary key index. Since there is no global write lock on the primary key index at this time, parallel write operations of multiple large fields can be supported.

[0063] In practical applications, the target data structure includes multiple leaf nodes; The step of determining the second data storage node corresponding to the second data in the target data structure based on the data index information of the second data includes: Based on the data index information of the second data and the index information of the target data structure, the second data storage node corresponding to the second data is determined from among the multiple leaf nodes contained in the target data structure.

[0064] The index information of the target data structure can be understood as the data index information stored in the row records of the leaf nodes in the target data structure.

[0065] Specifically, based on the data index information of the second data and the data index information of the row records stored in the leaf nodes of the target data structure, the second data storage node for writing the second data can be determined from among the multiple leaf nodes contained in the target data structure. This enables subsequent writing of the second data.

[0066] Furthermore, in one embodiment of this specification, it also includes: In response to the data deletion request for the first data, the first data stored in the first data storage node is deleted using a data deletion thread to obtain the first data storage node after deletion; An asynchronous allocation thread is used to provide the deleted first data storage node to the data writing thread for data writing.

[0067] Among them, the data deletion thread can be understood as the thread that deletes data, the asynchronous allocation thread can be understood as the thread that allocates external pages, and the data writing thread can be understood as the thread that writes the first and second data.

[0068] Specifically, when a data deletion request for large field data is received, the data deletion thread can be used to clean up the first data storage node, that is, to delete the first data stored in the first data storage node, obtain the deleted first data storage node, and use the asynchronous allocation thread to provide the deleted first data storage node to the data writing thread for subsequent data writing.

[0069] In practical applications, see Figure 7 , Figure 7 This diagram illustrates data deletion in a data processing method according to an embodiment of this specification. For data update and deletion operations, old external pages need to be cleaned up. To reduce the overhead of foreground write tasks, a background asynchronous allocation thread can be used for cleanup. The external pages of large fields marked for deletion can be prefetched from disk into memory. Then, while holding a file lock, the external page space is cleaned up and returned to the asynchronous allocation thread for reuse during the next large field data write. Since the data deletion task is not on the data write thread and the time-consuming disk read operation is performed without holding a global write lock, the parallel writing efficiency of large fields is improved.

[0070] In addition, after determining the data to be written carried in the data write request, the method further includes: If the data to be written is determined to be second data, the second data storage node corresponding to the second data is determined in the target data structure according to the data index information of the second data, wherein the data volume of the second data is less than the preset data volume threshold. Write the second data to the second data storage node.

[0071] Specifically, when the data to be written is not a large field, the second data storage node can be directly determined from the target data structure, and the non-large field data can be written to the second data storage node.

[0072] In the above method, after responding to a data write request for the target data structure, the data to be written carried by the data write request can be determined. If the data to be written contains the first data, since the data volume of the first data is greater than the preset data volume threshold, that is, the first data is a large field data, the first data cannot be written into the target data structure. The first data can be allocated to a first data storage node outside the target data structure, and the first data can be written to the external first data storage node when there is no lock on the target data structure. The first data is written without locking the first data storage node and without locking the target data structure, thus ensuring the writing performance of the first data.

[0073] The following is in conjunction with the appendix Figure 8 Taking the application of the data processing method provided in this specification for writing large-field data and non-large-field data as an example, the data processing method will be further explained. Figure 8 A flowchart illustrating the processing procedure of a data processing method according to an embodiment of this specification is shown, specifically including the following steps.

[0074] Step 802: In response to a data write request for a target data structure, determine the data to be written carried in the data write request.

[0075] Step 804: If it is determined that the data to be written contains both large field data and non-large field data, determine the second data storage node corresponding to the non-large field data in the target data structure according to the data index information of the non-large field data, and write the non-large field data into the second data storage node.

[0076] Specifically, in the first stage, based on the index information of the non-large field data, the position of the row record in the target data structure (i.e., the primary key index) can be located, and the content of the non-large field data can be modified. Holding a global read lock on the primary key index, starting from the root node of the target data structure, the search proceeds to the second data storage node where the large field data will be written. Then, a write lock is acquired on the second data storage node. The column data of the non-large field data stored in the primary key index is modified first, while reserving space for the column data of the large field data. Finally, the global read lock on the primary key index and the write lock on the second data storage node are released.

[0077] Step 806: Allocate a first data storage node outside the target data structure for the large field data, and write the large field data into the first data storage node.

[0078] Specifically, the second stage involves offline allocation and copying of the external pages (i.e., the first data storage node) for large field data. Without holding any locks on the primary key index (global locks and leaf node locks), the number of external pages required for each large field data column in the written row records is calculated. To improve space allocation efficiency, an asynchronous allocation thread is used to allocate the required external pages. The foreground thread writing the large field data only needs to allocate the disk space for the required external pages from the file system in batches while holding the file lock. For each large field data, the requested external pages are chained together into a page linked list, and the content of each large field data is copied one by one to the external page linked list. The content of the large field data that needs to be stored on the external pages is then written.

[0079] Step 808: Write the reference information of the first data storage node into the target data structure to realize the association between the first data storage node and the target data structure.

[0080] Specifically, the third stage involves modifying the external page pointers for large fields in the primary key index. In the first stage, because the content of the large fields was not written to the external pages, the row records only modified the non-large fields of the primary key index, reserving the space for pointing to external pages in each large field without actually writing to the actual external page location. Therefore, the global read lock on the primary key index and the write lock on the corresponding leaf node (i.e., the second data storage node) are held again. The positions of the external page linked lists corresponding to the already written large fields (i.e., the reference information of the first data storage node) are stored in the primary key index.

[0081] Step 810: In response to the data deletion request for large field data, use the background data deletion thread to delete the large field data stored in the first data storage node, and obtain the deleted first data storage node; use the asynchronous allocation thread to provide the deleted first data storage node to the data writing thread for the next data writing.

[0082] In this case, the data deletion request can mark only the large field data that needs to be deleted.

[0083] Specifically, a background asynchronous thread can be used for cleanup. First, the external pages of expired large field data are prefetched from the disk into memory. Then, the file lock is acquired, the external page space is cleaned up, and returned to the asynchronous allocation thread for reuse when allocating space for the next large field data write.

[0084] In the above method, after responding to a data write request for the target data structure, the data to be written carried by the data write request can be determined. If the data to be written contains the first data, since the data volume of the first data is greater than the preset data volume threshold, that is, the first data is a large field data, the first data cannot be written into the target data structure. The first data can be allocated to a first data storage node outside the target data structure, and the first data can be written to the external first data storage node when there is no lock on the target data structure. The first data is written without locking the first data storage node and without locking the target data structure, thus ensuring the writing performance of the first data.

[0085] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 9 A schematic diagram of the structure of a data processing apparatus according to one embodiment of this specification is shown. Figure 9 As shown, the device includes: The determination module 902 is configured to determine the data to be written carried in the data write request in response to a data write request for a target data structure. The allocation module 904 is configured to allocate a first data storage node outside the target data structure to the first data when it is determined that the data to be written contains first data, and the target data structure does not have a lock, wherein the amount of the first data is greater than a preset data amount threshold. The write module 906 is configured to write the first data to the first data storage node when there is no lock on the target data structure, wherein the first data storage node is associated with the target data structure.

[0086] In an optional embodiment, the allocation module 904 is further configured to: Based on the amount of the first data, a first data storage node other than the target data structure is allocated to the first data, wherein the storage space of the first data storage node is greater than or equal to the amount of data.

[0087] In an optional embodiment, the writing module 906 is further configured to: Determine the reference information corresponding to the first data storage node; Based on the reference information, the first data storage node and the target data structure are associated.

[0088] In an optional embodiment, the writing module 906 is further configured to: The reference information is stored in the target data structure, and the association between the first data storage node and the target data structure is realized based on the reference information stored in the target data structure.

[0089] In an optional embodiment, the writing module 906 is further configured to: If it is determined that the data to be written also contains second data, the second data storage node corresponding to the second data is determined in the target data structure according to the data index information of the second data, wherein the data volume of the second data is less than the preset data volume threshold; Write the second data to the second data storage node.

[0090] In an optional embodiment, the writing module 906 is further configured to: If a read lock exists in the target data structure, the second data storage node corresponding to the second data is determined in the target data structure based on the data index information of the second data. If a write lock exists on the second data storage node, the second data is written to the second data storage node.

[0091] In an optional embodiment, the writing module 906 is further configured to: Release the read lock on the target data structure and the write lock on the second data storage node.

[0092] In an optional embodiment, the writing module 906 is further configured to: If a read lock exists in the target data structure, the reference information is stored in the target data structure.

[0093] In one optional embodiment, the target data structure includes multiple leaf nodes; The writing module 906 is further configured as follows: Based on the data index information of the second data and the index information of the target data structure, the second data storage node corresponding to the second data is determined from among the multiple leaf nodes contained in the target data structure.

[0094] In an optional embodiment, the device further includes a deletion module configured to: In response to the data deletion request for the first data, the first data stored in the first data storage node is deleted using a data deletion thread to obtain the first data storage node after deletion; An asynchronous allocation thread is used to provide the deleted first data storage node to the data writing thread for data writing.

[0095] In an optional embodiment, the writing module 906 is further configured to: If the data to be written is determined to be second data, the second data storage node corresponding to the second data is determined in the target data structure according to the data index information of the second data, wherein the data volume of the second data is less than the preset data volume threshold. Write the second data to the second data storage node.

[0096] In the aforementioned device, in response to a data write request for a target data structure, the data to be written carried by the data write request can be determined. If the data to be written contains first data, and since the amount of the first data is greater than a preset data amount threshold, that is, the first data is large field data, the first data cannot be written into the target data structure. The first data can be allocated to a first data storage node outside the target data structure, and the first data can be written to the external first data storage node when there is no lock on the target data structure. The first data is written without locking the first data storage node and without locking the target data structure, thus ensuring the writing performance of the first data.

[0097] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0098] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0099] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0100] In one embodiment of this application, the aforementioned components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0101] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0102] The processor 1020 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0103] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the data processing method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data processing method embodiments.

[0104] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0105] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are basically similar to the data processing method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data processing method embodiments.

[0106] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0107] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0108] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0109] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0110] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0111] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0112] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: In response to a data write request for a target data structure, determine the data to be written carried in the data write request; If it is determined that the data to be written contains first data, and the target data structure does not have a lock, a first data storage node outside the target data structure is allocated to the first data, wherein the amount of the first data is greater than a preset data amount threshold. If no lock exists in the target data structure, the first data is written to the first data storage node, wherein the first data storage node is associated with the target data structure.

2. The method according to claim 1, wherein allocating a first data storage node outside the target data structure to the first data comprises: Based on the amount of the first data, a first data storage node other than the target data structure is allocated to the first data, wherein the storage space of the first data storage node is greater than or equal to the amount of data.

3. The method according to claim 1, further comprising, after writing the first data to the first data storage node: Determine the reference information corresponding to the first data storage node; Based on the reference information, the first data storage node and the target data structure are associated.

4. The method according to claim 3, wherein associating the first data storage node and the target data structure according to the reference information includes: The reference information is stored in the target data structure, and the association between the first data storage node and the target data structure is realized based on the reference information stored in the target data structure.

5. The method according to any one of claims 1-4, further comprising, after determining that the data to be written contains the first data: If it is determined that the data to be written also contains second data, the second data storage node corresponding to the second data is determined in the target data structure according to the data index information of the second data, wherein the data volume of the second data is less than the preset data volume threshold; Write the second data to the second data storage node.

6. The method according to claim 5, wherein determining the second data storage node corresponding to the second data in the target data structure based on the data index information of the second data comprises: If a read lock exists in the target data structure, the second data storage node corresponding to the second data is determined in the target data structure based on the data index information of the second data. The step of writing the second data to the second data storage node includes: If a write lock exists on the second data storage node, the second data is written to the second data storage node.

7. The method according to claim 6, further comprising, after writing the second data to the second data storage node when a write lock exists on the second data storage node: Release the read lock on the target data structure and the write lock on the second data storage node.

8. The method according to claim 4, wherein storing the reference information into the target data structure comprises: If a read lock exists in the target data structure, the reference information is stored in the target data structure.

9. The method according to claim 5, wherein the target data structure includes multiple leaf nodes; The step of determining the second data storage node corresponding to the second data in the target data structure based on the data index information of the second data includes: Based on the data index information of the second data and the index information of the target data structure, the second data storage node corresponding to the second data is determined from among the multiple leaf nodes contained in the target data structure.

10. The method according to any one of claims 1-4, further comprising: In response to the data deletion request for the first data, the first data stored in the first data storage node is deleted using a data deletion thread to obtain the first data storage node after deletion; An asynchronous allocation thread is used to provide the deleted first data storage node to the data writing thread for data writing.

11. The method according to any one of claims 1-4, wherein after determining the data to be written carried in the data write request, it further comprises: If the data to be written is determined to be second data, the second data storage node corresponding to the second data is determined in the target data structure according to the data index information of the second data, wherein the data volume of the second data is less than the preset data volume threshold. Write the second data to the second data storage node.

12. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11.

13. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 11.

14. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 11.

Images