Request processing methods and apparatus, electronic devices, computer-readable storage media

Abstract

This disclosure provides a request processing method, apparatus, electronic device, and computer-readable storage medium. The method includes: when a target data area of ​​a database is detected to meet the triggering conditions for a write request, obtaining a first write request currently triggered in the target data area, wherein the target data area is any data storage unit corresponding to the database; if a write request for the target data area already exists in a preset write request query queue, then determining that a target timed write request exists in the target data area; the target timed write request is a triggered and pending timed write request, and the target timed write request has a set delay time; setting the delay time of the target timed write request to zero, and the target timed write request is executed immediately; and storing the first write request in the write request query queue. According to the method of this disclosure, the probability of client write blocking to data areas in the database can be reduced, and the read / write performance of the database can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of storage technology, and in particular to a request processing method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] HBase is a distributed, column-oriented, open-source database storage system built on top of the Hadoop Distributed File System (HDFS). HBase is suitable for storing unstructured data. For storing large amounts of data, such as terabytes (TB) or higher, HBase offers significant improvements in data query and analysis efficiency compared to traditional relational databases.

[0003] In related technologies, a Region is the smallest unit of data management within HBase. When performing a write operation on a Region in HBase, the data is first stored in the Region's corresponding memory space. For example, when the memory size reaches a certain threshold, a flush operation is triggered on the memory space corresponding to that Region. The flush operation writes the data in the Region's memory space to disk space and clears the corresponding memory space. If the flush operation is not processed in a timely manner, it can easily cause client writes to HBase to be blocked, thereby affecting HBase's read and write performance. Summary of the Invention

[0004] This disclosure provides a request processing method and apparatus, an electronic device, and a computer-readable storage medium, which can reduce the probability of client write blocking to data areas in a database and improve database read and write performance.

[0005] In a first aspect, this disclosure provides a request processing method, which includes: when a target data area of ​​a database is detected to meet the triggering conditions for a flush request, obtaining a first flush request currently triggered in the target data area, wherein the target data area is any data storage unit corresponding to the database; if a flush request for the target data area already exists in a preset flush request query queue, then determining that a target timed flush request exists in the target data area; wherein the target timed flush request is a timed flush request that has been triggered and is pending processing, and the target timed flush request has a set delay time; setting the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately; and storing the first flush request in the flush request query queue.

[0006] Secondly, this disclosure provides a request processing apparatus, comprising: an acquisition module, configured to acquire a first flush request currently triggered in a target data area when the target data area of ​​a database is detected to meet the flush request triggering conditions, wherein the target data area is any data storage unit corresponding to the database; a determination module, configured to determine that a target timed flush request exists in the target data area if a flush request for the target data area already exists in a preset flush request query queue; wherein the target timed flush request is a timed flush request that has been triggered and is pending processing, and the target timed flush request has a set delay time; a setting module, configured to set the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately; and a storage module, configured to store the first flush request in the flush request query queue.

[0007] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the request processing method described above.

[0008] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-described request processing method when executed by a processor / processor core.

[0009] In the embodiments provided in this disclosure, when a flush request is triggered in the target data area of ​​the database, the first flush request currently triggered in the target data area is obtained. If it is determined that a flush request for the target data area already exists in the preset flush request query queue, it can be determined that there is a target timed flush request in the target data area, that is, a timed flush request that has been triggered and is pending before the first flush request. The delay time of the target timed flush request is set to zero, so that the target timed flush request becomes a request that needs to be executed immediately and is processed quickly. At this time, the first flush request is stored in the flush request query queue so that the first flush request is processed.

[0010] In related technologies, if a triggered and pending timed flush request is found in the target data area, it is necessary to wait for that timed flush request to be processed before processing a newly triggered timed flush request in the target data area can begin. In other words, if a triggered and pending timed flush request exists in the target data area, a newly triggered flush request in the data area will be ignored, which will lead to data write blocking in the target data area. Compared with related technologies, the method of this disclosure embodiment can preprocess the triggered and pending timed flush requests in the target data area, improve the flushing efficiency of the memory corresponding to the target data area in the database, reduce the probability of write blocking of the target data area in the database by the client, and thus improve the read and write performance of the database.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0013] Figure 1 A flowchart of a request processing method provided in an embodiment of this disclosure;

[0014] Figure 2 A flowchart of a request processing method provided as an exemplary embodiment of this disclosure;

[0015] Figure 3 A block diagram of a request processing apparatus provided in an embodiment of this disclosure;

[0016] Figure 4 A block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0018] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0019] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0020] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, they specify the presence of features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0021] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0022] In this embodiment, HBase is a columnar database based on Hadoop Distributed Computing; the underlying file system of HBase is the Hadoop Distributed File System (HDFS). HDFS is deployed on a server cluster consisting of multiple servers, each of which can be called a data node in HDFS. Data is stored in HBase in the form of tables. A data table in HBase is called an HBase table, which is a collection of rows, and rows are collections of column families. Each row is identified by a unique rowkey. An HBase table can be divided into multiple regions according to the range of the row keys, that is, divided into multiple regions along the row direction. A region can be considered a fragment of an HBase table.

[0023] A column family is a collection of columns, and a column is a collection of key-value pairs. The most basic unit in an HBase table is a column. A column family can be further divided into multiple data files. Data files are the underlying storage files for HBase data, also known as HFiles, and they are stored in the path of the source HBase cluster.

[0024] HBase's architecture consists of a master node and multiple region servers. Each region server manages one or more region objects (often referred to as HRegions), and each region object corresponds to a region in HDFS. The master is the service process deployed on the management node, while the region servers are the service processes deployed on the worker nodes. For ease of management, region servers are typically deployed on HDFS data nodes; correspondingly, HDFS data nodes also serve as HBase worker nodes.

[0025] HBase provides a client that can interact with RegionServers. When a user needs to access the HBase cluster, for example, to write data to or read data from the HBase cluster, it usually needs to send a corresponding request to the RegionServer in the HBase cluster through this client, and then the RegionServer will perform the corresponding data processing based on the request.

[0026] In some embodiments, a Region consists of one or more storage units (Stores). Each column family in an HBase table corresponds to a separate memory space, i.e., a Memstore; therefore, a Region can correspond to multiple memory spaces. After a write operation is performed on a Region, the written data is first stored in the column family's memory space. When the memory size reaches a certain threshold or data has been written to that memory for a certain period of time, the data in the Memstore is written to disk, forming an HFile file. This process is called a flush operation, also known as a cache write to disk operation or cache refresh operation. The flush operation is used to write the data in the memory space corresponding to the Region to the disk space to ensure that the data in the corresponding memory space is written to the data file and to clear the corresponding memory space.

[0027] In related technologies, HBase periodically refreshes the Memstore. For example, the default refresh cycle for the Memstore is 1 hour to ensure that the Memstore is not left unpersisted to disk for an extended period. To avoid all MemStores being flushed at the same time, periodic flush operations have a certain random delay (e.g., about 5 minutes), forming a timed flush request. Internally, each column family corresponds to a separate memory space, which is used to store the data of the current column family. The upper limit threshold of the memory space is 128MB by default. This upper limit threshold has a floating factor, which defaults to 2. The floating factor parameter is denoted as hbase.hregion.memstore.block.multiplier. In other words, client blocking only occurs when the memory size corresponding to the current column family reaches memstore*2 = 256MB. When a scheduled flush request is triggered, it writes data from memory to disk for the specified Region. This blocks all update operations on that Region, meaning write requests to that Region are blocked. If the client needs to write a large amount of data to the Region at this time (e.g., exceeding 256MB), it will cause write blocking. In this related technology, when data writes reach 256MB, if the previous scheduled flush request has not yet been processed, the current flush request for this region will be ignored. This will cause write blocking for the current region until the previous scheduled flush request is completely processed. Before the previous scheduled flush request is completely processed, the client's write blocking state significantly impacts HBase read and write performance.

[0028] This disclosure provides a request processing method that can improve the efficiency of flushing and writing memory corresponding to data areas in a database, reduce the probability of client write blocking to data areas in the database, and improve database read and write performance.

[0029] The request processing method according to embodiments of this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. Servers can include independent physical servers, server clusters consisting of multiple servers, or cloud servers capable of cloud computing. This request processing method can be implemented by a processor calling computer-readable program instructions stored in memory.

[0030] Figure 1 A flowchart illustrating a request processing method provided in an embodiment of this disclosure. (Refer to...) Figure 1 The request processing method may include the following steps.

[0031] S110: If the target data area of ​​the database is detected to meet the conditions for triggering a write request, the first write request currently triggered in the target data area is obtained. The target data area is any data storage unit corresponding to the database.

[0032] S120, if a brush request for the target data area already exists in the preset brush request query queue, then it is determined that a target timed brush request exists for the target data area; wherein, the target timed brush request is a timed brush request that has been triggered and is pending processing, and the target timed brush request is set with a delay time.

[0033] S130, set the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately.

[0034] S140, store the first write request in the write request query queue.

[0035] In this embodiment of the disclosure, when a flush request is triggered in the target data area of ​​the database, the first flush request currently triggered in the target data area is obtained. If it is determined that a flush request for the target data area already exists in the preset flush request query queue, it can be determined that there is a target timed flush request in the target data area, that is, a timed flush request that has been triggered and is pending before the first flush request. The delay time of the target timed flush request is set to zero, so that the target timed flush request becomes a request that needs to be executed immediately and is processed quickly. At this time, the first flush request is stored in the flush request query queue so that the first flush request is processed.

[0036] In related technologies, if a triggered and pending timed flush request is found in the target data area, it is necessary to wait for that timed flush request to be processed before processing a newly triggered timed flush request in the target data area can begin. In other words, if a triggered and pending timed flush request exists in the target data area, a newly triggered flush request in the data area will be ignored, which will lead to data write blocking in the target data area. Compared with related technologies, the method of this disclosure embodiment can preprocess the triggered and pending timed flush requests in the target data area, improve the flushing efficiency of the memory corresponding to the target data area in the database, reduce the probability of write blocking of the target data area in the database by the client, and thus improve the read and write performance of the database.

[0037] In this embodiment, when the delay time of the target timed flush request is set to zero, the target timed flush request will be processed immediately, thereby reducing client write blocking. It should be understood that if the delay time is not set to zero, it is uncertain whether client write blocking can be reduced. For example, if the target timed flush request for the target data area still needs n seconds to be processed, where n is a value greater than 0; when n = 60, the client's write request will still be blocked for 1 minute before execution; however, when the delay time of the target timed flush request is set to 0, the target timed flush request will be processed immediately, thereby reducing the probability of client write blocking to the database.

[0038] In the following description of the embodiments, HBase is used as an example to describe the request processing method of the present disclosure. It should be understood that the database can also be other distributed computing databases, and the present disclosure does not specifically limit it.

[0039] In this embodiment of the disclosure, when HBase triggers a flush request, the RegionServer can start a thread to process the flush request. This thread can be referred to as the flush thread. The processing logic of the flush thread can be implemented using the relevant code of the MemStoreFlusher function class. In the processing logic implemented through the MemStoreFlusher function class, timed flush requests, i.e., timed write requests, are mainly handled by the requestDelayedFlush function. This method is used to collect flush requests and related information, for example, it can be used to store timed flush requests into a specified queue.

[0040] In some embodiments, the processing of timed flush requests involves three queues: a region information queue (regionsInQueue), a flush queue (flushQueue), and a flush request query queue (randomDelayQueue). The first two queues, regionsInQueue and flushQueue, are queues existing in related technologies, while the last queue, randomDelayQueue, is a newly added queue in this embodiment to address the write blocking problem. The flushQueue is a blocking queue, which can be understood as a flush working queue, storing timed flush requests and waiting for processing by the flush thread. The regionsInQueue stores information about the data storage unit to which the timed flush requests located in the flushQueue belong. Since one region can correspond to multiple memory spaces, one region can correspond to one or more flush requests.

[0041] In this embodiment of the disclosure, the request processing method can be implemented by the flush thread calling the flush request processing method (i.e., the requestFlush method).

[0042] In some embodiments, regionsInQueue stores information about the data storage unit to which the delayed flush request belongs in flushQueue. Therefore, the requestFlush method can access regionsInQueue to determine if there are any unprocessed flush requests in the target Region. Specifically, if information about the target Region exists in regionsInQueue, it indicates that there are unprocessed flush requests in the target Region.

[0043] In some embodiments, the flushQueue stores timed flush requests waiting to be processed by the flush thread. When a flush request is stored in the flushQueue, a random delay time is set for it, resulting in a timed flush request. Timed flush requests in the flushQueue can only be retrieved from the flushQueue when the set delay time has elapsed, thus enabling the execution of the operation to write the data in the memory corresponding to the target area to the disk space.

[0044] In this embodiment of the disclosure, randomDelayQueue is a custom queue newly added in this embodiment of the disclosure. The updates of the three queues involved in this embodiment of the disclosure, regionsInQueue, randomDelayQueue and flushQueue, are synchronized. If there are unprocessed flush requests in the target Region, the Region information of the target Region needs to be stored in regionsInQueue, and the flush requests of the target Region (flush requests without a random delay time or flush requests with a random delay time can be stored in randomDelayQueue) need to be stored in randomDelayQueue. The timed flush request obtained after setting the delay time of the flush request is stored in flushQueue.

[0045] In some embodiments, after obtaining the first write request currently triggered by the target data area in step S110, the request processing method further includes: S11, if the target data area information is not present in the preset area information queue, the target data area information is written into the area information queue, and the first write request is processed according to a predetermined procedure; wherein, the area information queue is used to store the information of the data storage unit to which the timed write request belongs in the write queue, and the write queue is the storage queue for the timed write request.

[0046] In this embodiment, the pre-processing includes: storing the first write request in the write request query queue; setting a delay time for the first write request to obtain a first timed write request; and storing the first timed write request in the write queue.

[0047] In this embodiment, the updates of the three queues, regionsInQueue, randomDelayQueue, and flushQueue, are synchronized. In step S11, if the target data region information is not found in regionsInQueue, it indicates that the target Region has no unprocessed flush requests. Therefore, it is necessary to update all three queues, regionsInQueue, randomDelayQueue, and flushQueue, based on the first flush request triggered by the target Region. Specifically, the target Region information needs to be stored in regionsInQueue, the first flush request needs to be stored in randomDelayQueue, and a random delay needs to be set for the first flush request. The resulting first timed flush request is then stored in flushQueue, providing a query basis and data foundation for the processing of subsequent flush requests.

[0048] In some embodiments, the request processing method further includes: S12, if there is information about the target data region in the region information queue and there is no brush request for the target data region in the brush request query queue, then perform predetermined processing on the first brush request.

[0049] In this embodiment, if the target Region information exists in the region information queue, but the flush request query queue does not contain any flush requests for the target data region, it indicates that there are unprocessed flush requests for the target Region. However, these unprocessed flush requests do not include the first flush request currently triggered by the target Region. Therefore, the randomDelayQueue and flushQueue can be updated based on the first flush request through the aforementioned pre-defined processing. Specifically, the first flush request needs to be stored in randomDelayQueue, and a random delay needs to be set for the first flush request. The obtained first timed flush request is then stored in flushQueue to provide a query basis and data foundation for the processing of subsequent flush requests.

[0050] In some embodiments, before step S130, the request processing method further includes: determining the storage location of the target timed write request in a preset write queue; wherein, the write requests in the write request query queue correspond one-to-one with the timed write requests in the write queue.

[0051] In this embodiment, step S130 may specifically include: S41, deleting the target timed flush request from the flush queue; S42, obtaining the flush request corresponding to the target timed flush request from the flush request query queue, setting the delay time of the corresponding flush request to zero, and obtaining the second flush request; S43, storing the second flush request into the flush queue; S44, obtaining the second flush request from the flush queue, and immediately executing the operation of writing the data in the memory corresponding to the target data area to the disk space.

[0052] For example, if a flush request exists in the randomDelayQueue, it means that the target Region has a target timed flush request (which is waiting to be processed in the flushQueue). In this case, it is necessary to modify the delay time of the target timed flush request. Since the flushQueue does not support the operation of obtaining requests, in order to ensure the smooth execution of the delay time modification operation, in this embodiment of the disclosure, the target timed flush request can be directly deleted from the flushQueue, and the flush request corresponding to the target timed flush request can be obtained from the randomDelayQueue. Then, the delay time of the corresponding flush request is set to 0 to obtain the second flush request. After the setting is completed, this second flush request is stored in the flushQueue. At this time, the second flush request in the flushQueue can be retrieved, and the operation of writing the data in the memory corresponding to the target data region to the disk space can be executed immediately, thereby improving the region flushing efficiency of the database, reducing the probability of client write blocking on HBase, and improving the read and write performance of HBase.

[0053] In some embodiments, after step S43, the request processing method further includes: deleting the information of the target data region from a preset region information queue, and deleting the corresponding flash request from a flash request query queue.

[0054] In this embodiment, after the target timed flush request is retrieved from the flushQueue and executed immediately, the target timed flush request no longer exists in the flushQueue. At this time, the information of the target data region corresponding to the regionInQueue and the flush request corresponding to the target timed flush request in the randomDelayQueue can be deleted together to ensure that the information of the above three queues is synchronized, so as not to interfere with subsequent request processing and to ensure the smooth execution of request processing logic.

[0055] In some embodiments, after obtaining the first write request currently triggered in the target data area in step S110, the request processing method further includes: waiting for a predetermined buffer duration and then determining whether there is a write request for the target data area in the preset write request query queue; wherein the predetermined buffer duration is less than or equal to a predetermined duration threshold.

[0056] In this embodiment, when a flush request is triggered for the target data area of ​​the database, after waiting for a predetermined buffer time, it is determined whether a flush request for the target data area already exists in the preset query queue. This allows for buffer processing time for the triggered and pending timed flush requests for the target data area, so that the target timed flush requests for the target data area (i.e., the triggered and pending timed flush requests) can be completed within the predetermined buffer time. This avoids frequent queue access and judgment operations and improves the flexibility of the request processing method.

[0057] In some embodiments, the conditions for triggering a write request include: the actual storage capacity of the memory corresponding to the target data region meets a preset storage capacity threshold; when the target data region of the database is detected to meet the write request conditions, the request processing method further includes: S51, obtaining a first target storage capacity of the memory corresponding to the target data region, wherein the first target storage capacity is the upper limit of the storage capacity of the memory corresponding to the target data region, and the first target storage capacity is determined according to a preset storage capacity threshold and a preset storage capacity multiple; S52, expanding the storage capacity of the memory corresponding to the target storage region according to the first target storage capacity to obtain a second target storage capacity; S53, when the actual storage capacity of the memory corresponding to the target data region meets the second target storage capacity, determining that the state of the memory corresponding to the target data region is a write blocking state.

[0058] For example, the first target storage capacity is, for instance, 128MB. After expanding the storage capacity of the memory corresponding to the target storage region based on the first target storage capacity, the second target storage capacity is, for instance, 512MB. When the actual storage capacity (size_memstore) of the memory corresponding to the current Region reaches 128MB, a flush request is triggered. However, the client will not be blocked at this time. The client will only truly enter the write blocking state when the data size of the memory corresponding to the current Region reaches 512MB. Therefore, as long as the target periodic flush request for the target data region is processed when the data is written to 512MB, the client will not be blocked. This helps to prevent the client from entering the write blocking state too quickly, reducing the system processing burden and improving system performance.

[0059] In some embodiments, step S52 may specifically include: expanding the storage capacity of the target storage region according to the first target storage capacity and the floating parameter to obtain a second target storage capacity, wherein the floating parameter is greater than a multiple of the storage capacity.

[0060] For example, if the preset storage capacity threshold is 128MB and the storage capacity multiplier is 1, then the first target storage capacity is, for example, 128MB. At this time, if the floating parameter (hbase.hregion.memstore.block.multiplier) is set to 4, then the second target storage capacity is 128MB*4=512MB.

[0061] In this embodiment, by setting a floating parameter, when a flush request is triggered by a target data area in the database, the upper limit threshold of the memory corresponding to the target data area can be adjusted. For example, if the default value of the floating parameter is 2, this embodiment can adjust the floating parameter so that the adjusted floating parameter is greater than the default value, for example, 4. This method can effectively control excessively fast client writes during HBase data writing, avoiding client write blocking due to excessive speed, thereby reducing the system processing burden and improving system performance and the flexibility of the flush request processing method. It should be understood that the floating parameter can be customized according to actual conditions, and this embodiment does not impose specific limitations.

[0062] According to the request processing method of this disclosure, the triggered and pending timed write requests for the target data area can be processed in advance, thereby improving the write efficiency of the memory corresponding to the target data area in the database, reducing the probability of client write blocking to the target data area in the database, and thus improving the read and write performance of the database.

[0063] Figure 2 A detailed flowchart of a request processing method according to an exemplary embodiment of this disclosure is provided. Figure 2 As shown, the request processing method includes the following steps.

[0064] S210, the client sends a write data request to the RegionServer in the HBase cluster.

[0065] S220, the regional server receives the data write request and performs the corresponding processing (Handle).

[0066] Specifically, refer to Figure 2 Step S220 may include the following steps.

[0067] S221, the region server writes the data carried in the data write request into the memory space of the specified region inside HBase.

[0068] S222, if the target data area meets the conditions for triggering a write request, obtain the first write request currently triggered in the target data area.

[0069] S223, if a brush request for the target data area already exists in the brush request query queue, determine that a target timed brush request exists for the target data area.

[0070] Among them, the target timed flush request is the timed flush request that has been triggered and is yet to be processed in the target data area.

[0071] S224, set the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately.

[0072] S225, store the first write request in the write request query queue.

[0073] According to the request processing method of this disclosure, compared with the prior art, when there are already unprocessed delayed flush requests, the flush request of the target data region will be ignored, resulting in the write blocking of the current region data. The method of this disclosure can improve the flush efficiency of the memory corresponding to the data region in the database, reduce the probability of the client writing to the database, and thus help improve the read and write performance of the database.

[0074] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0075] In addition, this disclosure also provides a request processing apparatus, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the request processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0076] Figure 3 This is a block diagram of a request processing apparatus provided according to an embodiment of the present disclosure. (Refer to...) Figure 3 This disclosure provides a request processing apparatus 300, which may include:

[0077] The acquisition module 310 is used to acquire the first flush request currently triggered in the target data area when the target data area of ​​the database is detected to meet the flush request triggering conditions. The target data area is any data storage unit corresponding to the database.

[0078] The determination module 320 is used to determine if there is a target timed flush request for the target data area if there is already a flush request for the target data area in the preset flush request query queue; wherein, the target timed flush request is a timed flush request that has been triggered and is waiting to be processed, and the target timed flush request is set with a delay time.

[0079] Setting module 330 is used to set the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately;

[0080] Storage module 340 is used to store the first write request into the write request query queue.

[0081] In some embodiments, the request processing device 300 further includes: a processing module, configured to, after obtaining a first write request currently triggered in the target data area, if the target data area information is not present in a preset area information queue, write the target data area information into the area information queue and perform predetermined processing on the first write request; wherein, the area information queue is used to store information of the data storage unit to which the timed write request belongs in the write queue, and the write queue is a storage queue for timed write requests. The predetermined processing includes: storing the first write request in a write request query queue; setting a delay time for the first write request to obtain a first timed write request; and storing the first timed write request in the write queue.

[0082] In some embodiments, the processing module is further configured to perform predetermined processing on the first write request if the information of the target data region exists in the region information queue and the write request query queue does not contain a write request for the target data region.

[0083] In some embodiments, the determining module 320 is further configured to determine that the storage location of the target timed write request is in a preset write queue before setting the delay time of the target timed write request to zero so that the target timed write request is executed immediately; wherein, the write requests in the write request query queue correspond one-to-one with the timed write requests in the write queue.

[0084] In this embodiment, the setting module 330 is further configured to delete the target timed write request from the write queue; obtain the write request corresponding to the target timed write request from the write request query queue, set the delay time of the corresponding write request to zero, and obtain the second write request; store the second write request in the write queue; obtain the second write request from the write queue, and immediately execute the operation of writing the data in the memory corresponding to the target data area to the disk space.

[0085] In some embodiments, the request processing device 300 further includes a deletion module, configured to delete information about the target data region from a preset region information queue after storing the second write request in the write queue, and to delete the corresponding write request from the write request query queue.

[0086] In some embodiments, the request processing device 300 further includes a buffer waiting module, configured to, after obtaining the first flush request currently triggered in the target data area, wait for a predetermined buffer duration and then determine whether a flush request for the target data area already exists in a preset flush request query queue; wherein the predetermined buffer duration is less than or equal to a predetermined duration threshold.

[0087] In some embodiments, the conditions for triggering a write request include: the actual storage capacity of the memory corresponding to the target data region meets a preset storage capacity threshold; when the target data region of the database is detected to meet the write request conditions, a first target storage capacity acquisition module is used to acquire a first target storage capacity of the memory corresponding to the target data region, wherein the first target storage capacity is the upper limit of the storage capacity of the memory corresponding to the target data region, and the first target storage capacity is determined according to the preset storage capacity threshold and a preset storage capacity multiple; a second target storage capacity acquisition module is used to expand the storage capacity of the memory corresponding to the target storage region according to the first target storage capacity to obtain a second target storage capacity; and a blocking state determination module is used to determine that the state of the memory corresponding to the target data region is a write blocking state when the actual storage capacity of the memory corresponding to the target data region meets the second target storage capacity.

[0088] According to the request processing apparatus of this application embodiment, when a flush request is detected to be triggered in the target data area of ​​the database, the first flush request currently triggered in the target data area is obtained; if it is determined that a flush request for the target data area already exists in the preset flush request query queue, it can be determined that there is a target timed flush request in the target data area, that is, a timed flush request that has been triggered and is pending processing before the first flush request. The delay time of the target timed flush request is set to zero, so that the target timed flush request becomes a request that needs to be executed immediately and is processed quickly. At this time, the first flush request is stored in the flush request query queue so that the first flush request is processed.

[0089] In related technologies, if a triggered and pending timed flush request is found in the target data area, it is necessary to wait for that timed flush request to be processed before processing a newly triggered timed flush request in the target data area can begin. In other words, if a triggered and pending timed flush request exists in the target data area, a newly triggered flush request in the data area will be ignored, which will lead to data write blocking in the target data area. Compared with related technologies, the method of this disclosure embodiment can preprocess the triggered and pending timed flush requests in the target data area, improve the flushing efficiency of the memory corresponding to the target data area in the database, reduce the probability of write blocking of the target data area in the database by the client, and thus improve the read and write performance of the database.

[0090] It should be clarified that this disclosure is not limited to the specific configurations and processes described in the foregoing embodiments and shown in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here, and the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0091] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0092] Reference Figure 4 This disclosure provides an electronic device, which includes: at least one processor 401; at least one memory 402; and one or more I / O interfaces 403 connected between the processor 401 and the memory 402; wherein the memory 402 stores one or more computer programs that can be executed by the at least one processor 401, and the one or more computer programs are executed by the at least one processor 401 to enable the at least one processor 401 to perform the above-described request processing method.

[0093] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the request processing method described above. The computer-readable storage medium may be volatile or non-volatile.

[0094] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described request processing method.

[0095] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0096] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0097] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0098] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0099] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0100] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0101] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0102] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0104] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A request processing method, characterized in that, include: If the target data area of ​​the database is detected to meet the conditions for triggering a flush request, the first flush request currently triggered in the target data area is obtained, where the target data area is any data storage unit corresponding to the database. If a flush request for the target data area already exists in the preset flush request query queue, then it is determined that a target timed flush request exists for the target data area; wherein, the target timed flush request is a timed flush request that has been triggered and is pending processing, and the target timed flush request is set with a delay time. Set the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately; The first write request is stored in the write request query queue.

2. The method according to claim 1, characterized in that, After obtaining the first write request currently triggered in the target data region, the method further includes: If the target data region is not present in the preset region information queue, the target data region is written into the region information queue, and the first write request is processed according to a predetermined procedure. The region information queue is used to store information about the data storage unit to which the timed write request belongs in the write queue. The write queue is a storage queue for timed write requests. The predetermined process includes: Store the first write request into the write request query queue; Set a delay time for the first write request to obtain the first timed write request; The first timed write request is stored in the write queue.

3. The method according to claim 2, characterized in that, The method further includes: If the target data region information exists in the region information queue, and the flash request query queue does not contain a flash request for the target data region, then the predetermined processing is performed on the first flash request.

4. The method according to claim 1, characterized in that, Before setting the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately, the method includes: determining that the storage location of the target timed flush request is in a preset flush queue; wherein, the flush requests in the flush request query queue correspond one-to-one with the timed flush requests in the flush queue; Setting the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately includes: Remove the target timed flush request from the flush queue; Obtain the flash request corresponding to the target timed flash request from the flash request query queue, set the delay time of the corresponding flash request to zero, and obtain the second flash request; Store the second write request in the write queue; The second write request is retrieved from the write queue, and the operation of writing the data in the memory corresponding to the target data area to the disk space is executed immediately.

5. The method according to claim 4, characterized in that, After storing the second write request in the write queue, the method further includes: The information of the target data region is deleted from the preset region information queue, and the corresponding flash request is deleted from the flash request query queue.

6. The method according to claim 1, characterized in that, After obtaining the first write request currently triggered in the target data region, the method further includes: After waiting for a predetermined buffer duration, it is determined whether a write request for the target data area already exists in the preset write request query queue; wherein, the predetermined buffer duration is less than or equal to a predetermined duration threshold.

7. The method according to claim 1, characterized in that, The conditions for triggering the write request include: the actual storage capacity of the memory corresponding to the target data area meets a preset storage capacity threshold. If the target data area of ​​the database is detected to meet the conditions for a write request, the method further includes: Obtain the first target storage capacity of the memory corresponding to the target data region. The first target storage capacity is the upper limit of the storage capacity of the memory corresponding to the target data region, and the first target storage capacity is determined according to the preset storage capacity threshold and the preset storage capacity multiple. The storage capacity of the memory corresponding to the target data area is expanded according to the first target storage capacity to obtain the second target storage capacity; If the actual storage capacity of the memory corresponding to the target data area meets the second target storage capacity, the state of the memory corresponding to the target data area is determined to be a write blocking state.

8. The method according to claim 7, characterized in that, The step of expanding the storage capacity of the memory corresponding to the target data region according to the first target storage capacity to obtain the second target storage capacity includes: The storage capacity of the target data area is expanded based on the first target storage capacity and the floating parameter to obtain a second target storage capacity, wherein the floating parameter is greater than the storage capacity multiple.

9. A brush request processing device, characterized in that, include: The acquisition module is used to acquire the first flush request currently triggered in the target data area when the target data area of ​​the database is detected to meet the flush request triggering conditions. The target data area is any data storage unit corresponding to the database. The determination module is used to determine that if a brush request for the target data area already exists in a preset brush request query queue, then the target data area has a target timed brush request; wherein the target timed brush request is a timed brush request that has been triggered and is pending processing, and the target timed brush request is set with a delay time. The setting module is used to set the delay time of the target timed flush request to zero so that the target timed flush request is executed immediately. The storage module is used to store the first write request into the write request query queue.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the request processing method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the request processing method as described in any one of claims 1-8.

12. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying said computer-readable code, characterized in that, When the computer-readable code is run in a processor of an electronic device, the processor performs the request processing method as described in any one of claims 1-8.

Images