Access request processing method, database, node of database, equipment, medium and product

Abstract

The embodiment of the invention provides an access request processing method, a database, a node of the database, a medium and a product. The method comprises the steps that an access node in the database can firstly determine the size of a to-be-processed data packet which serves as network data and is stored in a buffer area of the database; when the internal memory pressure of the database is large, the access node can control the data writing amount of the internal memory of the database, that is to say, the to-be-processed data packet with the large data amount is controlled to be continuously stored in the buffer area of the database and is not read into the internal memory of the database. In the method, the access node can control the data packet with a large data volume to be continuously stored in the buffer area instead of being directly discarded, that is, the data write-in amount and write-in speed of the database memory are controlled, so that a user does not obtain error report information which is generated due to discarding of the data packet and reflects the access failure of the database, and the user experience is improved. While the pressure in the database is reduced, the success rate of database access is improved, and the user experience is also ensured.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to an access request processing method, a database and its nodes, devices, media and products. Background Technology

[0002] The database access node is the first point of contact for data packets generated by User Equipment (UE) arriving at the database. This access node performs a series of processes on the data packets, including authentication, parsing, and metadata management, before reading the processed packets into the database's memory. The data packets in the database memory can then be further processed. In practice, the access node may receive a large number of concurrent data packets, which can significantly increase the database's memory pressure.

[0003] To reduce the memory pressure on the database, the amount of data written to the database memory can be controlled, but how to control it reasonably becomes an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an access request processing method, a database and its nodes, devices, media and products, for reasonably controlling the amount of data written to the database memory.

[0005] In a first aspect, embodiments of the present invention provide an access request processing method, including:

[0006] Determine the size of the data packet to be processed in the buffer of the database access request, wherein the data packet to be processed is contained in the database access request and the access request contains the size information of the data packet to be processed;

[0007] Based on the size information of the data packet to be processed, determine whether to read the data packet to be processed from the buffer;

[0008] If the size of the data packet to be processed is not less than a preset threshold, and the current actual write status of the database does not meet the current reference write requirements of the database, then the data packet to be processed is controlled to continue to be stored in the buffer.

[0009] Secondly, embodiments of the present invention provide a database node, wherein a physical device on which a database is deployed allocates a buffer and memory for the database node;

[0010] The database node is used to determine the size of the data packet to be processed in the buffer. The data packet to be processed is contained in the database access request, and the access request contains the size information of the data packet to be processed.

[0011] Based on the size of the data packet to be processed, determine whether to read the data packet from the buffer;

[0012] If the size of the data packet to be processed is not less than a preset threshold, and the current actual write status of the database does not meet the current reference write requirements of the database, then the data packet to be processed is controlled to continue to be stored in the buffer.

[0013] Thirdly, embodiments of the present invention provide a database deployed in a physical device, the database including at least one access node; the physical device allocates buffers and memory for any access node in the database;

[0014] The access node is used to determine the size of the data packet to be processed in the buffer, the data packet to be processed is contained in the access request of the database, and the access request contains the size information of the data packet to be processed;

[0015] Based on the size of the data packet to be processed, determine whether to read the data packet to be processed from the buffer and write it into the memory;

[0016] If the size of the data packet to be processed is not less than a preset threshold, and the current actual write status of the database does not meet the current reference write requirements of the database, then the data packet to be processed is controlled to continue to be stored in the buffer.

[0017] Fourthly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the access request processing method described in the first aspect. The electronic device may also include a communication interface for communicating with other devices or communication networks.

[0018] Fifthly, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, wherein when the executable code is executed by a processor of an electronic device, the processor is able to at least implement the access request processing method as described in the first aspect.

[0019] In a sixth aspect, embodiments of the present invention provide a computer program product, the computer program product including a computer program or instructions, which, when executed by a processor, cause the processor to implement the access request processing method as described in the first aspect above.

[0020] The access request processing method provided in this embodiment of the invention allows the access node in the database to first determine the size of the data packet to be processed. This data packet, which is network data, is included in the database access request, which also includes the size information of the data packet. The data packet can be sent from the user device's buffer and written into the database access request buffer by the access node. For the data packet in the buffer, the access node then determines whether to control the amount of data written to the database memory based on the size information of the data packet. When the size of the data packet is not less than a preset threshold, and the current actual write situation of the database does not meet the current reference write requirements, it indicates that the database memory is tight. In this case, the access node can control the amount of data written to the database memory, that is, continue to store the data packet in the database buffer instead of reading it from the buffer into the database memory.

[0021] The above method provides a way to control the amount of data written to memory. The access node can control the continued storage of large data packets in the buffer, thereby reducing the data write speed and volume to the database memory and thus reducing memory pressure. Furthermore, compared to reducing memory pressure by discarding data packets, this method avoids errors indicating database access failures caused by packet dropping. Therefore, the flow control method provided by this approach is more reasonable, as it can reduce database memory pressure while also improving the success rate of database access and ensuring a better user experience. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of an access request processing method provided in an embodiment of the present invention;

[0024] Figure 2 A schematic diagram illustrating a data write volume control process provided in an embodiment of the present invention;

[0025] Figure 3 A flowchart of another access request processing method provided in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram illustrating an iterative adjustment process of token bucket parameters provided in an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the structure of a physical device provided in an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the structure of another physical device provided in an embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the structure of an access request processing device provided in an embodiment of the present invention;

[0030] Figure 8 To and Figure 7 The illustrated embodiment provides a schematic diagram of the electronic device corresponding to the access request processing device. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0033] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0034] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to identification.” Similarly, depending on the context, the phrases “if determination” or “if identification (of the condition or event of the statement)” can be interpreted as “when determination” or “in response to determination” or “when identification (of the condition or event of the statement)” or “in response to identification (of the condition or event of the statement).”

[0035] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0036] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0037] Before describing the various embodiments provided by the present invention in detail, the application scenarios of the present invention can also be introduced in conjunction with the content of the background art.

[0038] The background technology mentions that the access node of the database can receive data packets generated by the UE. In the various embodiments provided by this invention, the data packet can be considered as network data transmitted through a communication channel between the UE and the database, such as a socket channel. Therefore, the specific process of the data packet being transmitted from the UE to the access node can be described as follows: the data packet generated by the UE can first be stored in a buffer provided by the network communication device configured in the UE. Then, the data packet can be transmitted from the UE's buffer to the buffer provided by the network communication device configured in the database. Finally, the access node can obtain the data packet generated by the UE from the database's buffer. Optionally, the network communication devices configured for the UE and the database can be any hardware or software that implements network data transmission, such as a network interface card (NIC). Optionally, the database can be an analytical database or a transactional database.

[0039] In the various embodiments provided by this invention, the access node can further write data packets from the database buffer to the database memory, so that other nodes in the database can process the data packets in memory, that is, complete the response to the data packets. In practice, considering the size and number of data packets read into memory, the database memory may experience excessive pressure, and increased memory pressure will further affect the performance of the database, such as the database being unable to process data packets in a timely manner, resulting in slow database access or even complete paralysis. Optionally, memory pressure can be reflected by memory utilization. Optionally, when the memory utilization is greater than a preset percentage, the memory pressure can be considered high. The preset percentage can be 100%, or it can be a percentage slightly less than 100%, such as 80%, 90%, etc.

[0040] In order to reduce the memory pressure on the database and ensure its performance, the methods provided in the following embodiments of the present invention can be used.

[0041] Based on the above description, some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0042] Figure 1 This is a flowchart illustrating an access request processing method provided in an embodiment of the present invention. This method provided in this embodiment can be executed by a database access node. Figure 1 As shown, the method may include the following steps:

[0043] S101, determine the size of the data packet to be processed in the database access request buffer. The data packet to be processed is contained in the database access request, and the access request contains the size information of the data packet to be processed.

[0044] The access node in the database can first determine the size of the data packets to be processed in the buffer. Optionally, since the data packets to be processed may correspond to the access requests generated by the UE's user to the database, the buffer mentioned in the embodiments of the present invention is also the buffer used in the database to store access requests, and can also be called the database access request buffer. Optionally, the access requests in the embodiments of the present invention are more often write requests to the database. Of course, optionally, the access requests may also be requests for deleting, modifying, and querying data in the database. And after the data packet processing is completed, the response to the access request is also completed.

[0045] To determine the size of data packets, one option is for the access node to read the data packets to be processed into the database memory and parse them. The size of the data packets can then be determined based on the parsing results. Furthermore, to prevent the data packets from consuming excessive memory, they can be deleted from memory after parsing, thus releasing the memory space used for data packet parsing.

[0046] S102, based on the size information of the data packet to be processed, determine whether to read the data packet to be processed from the buffer.

[0047] S103, if the size of the data packet to be processed is not less than the preset threshold, and the current actual writing situation of the database does not meet the current reference writing requirements of the database, then control the data packet to be processed to continue to be stored in the buffer.

[0048] Then, based on the size of the data packet to be processed, the access node can further determine whether to write the data packet to be processed from the database buffer to the database memory. That is, the size of the data packet to be processed is used as a basis to determine whether to control the amount of data written to the database memory.

[0049] One scenario is as follows: If the size of the data packet to be processed is not less than a preset threshold, and the current actual write volume of the database does not meet the current reference write requirements, it indicates that the data volume of the data packet to be processed is large. This also suggests that directly writing this large data packet to memory may cause the database memory pressure to exceed the preset pressure limit. In this case, the access node can control the data packet to be processed to continue to be stored in the buffer, which means controlling the amount of data written to the database memory.

[0050] Optionally, the current actual write status of the database, i.e., the latest write status of the database when the access node determines the size of the data packet to be processed, reflects the actual pressure on the database memory. The current reference write requirement of the database, i.e., the latest write requirement when the size of the data packet to be processed is determined, reflects the maximum pressure that the database memory can withstand while ensuring database performance, i.e., the aforementioned preset pressure limit. As can be seen from the above description, optionally, memory pressure can be reflected by memory utilization, and the preset limit can be a preset utilization ratio, such as 100%, 80%, 90%, etc.

[0051] In this embodiment, the access node in the database can first determine the size of the data packet to be processed. Then, based on the size of the data packet, the access node determines whether to control the amount of data written to the database memory. When the size of the data packet to be processed is not less than a preset threshold, and the current actual write situation of the database does not meet the current reference write requirements of the database, it indicates that the current database memory is tight. At this time, the access node can control the amount of data written to the database memory, that is, continue to store the data packet to be processed in the database buffer, instead of reading the data packet to be processed into the database memory.

[0052] The above method provides a way to control the amount of data written to memory. The access node can control the continued storage of large data packets in the buffer, thereby reducing the data write speed and volume to the database memory and thus reducing memory pressure. Furthermore, compared to reducing memory pressure by discarding data packets, this method avoids user errors indicating database access failures caused by packet dropping. Therefore, the flow control method provided by this approach is more reasonable, improving database access success rate while reducing database memory pressure and ensuring a better user experience.

[0053] In addition, Figure 1 The embodiment shown in step S103 provides one scenario, but another scenario is: if the size of the data packet to be processed is less than the preset threshold, it indicates that the amount of data in the data packet to be processed is small, and writing the data packet to be processed is unlikely to cause the memory pressure of the database to exceed the preset pressure limit. In this case, the access node can directly read the data packet to be processed from the buffer and write it into the memory of the database.

[0054] Another scenario is: if the size of the data packet to be processed is not less than the preset threshold, and if the current actual writing situation of the database meets the current reference writing requirements of the database, it indicates that the data volume of the data packet to be processed is large, and it also indicates that if the large data packet to be processed is directly written into memory, it will not cause the memory pressure of the database to exceed the preset pressure limit. In this case, the access node can directly read the data packet to be processed from the buffer and write it into the memory of the database.

[0055] In this embodiment, with Figure 1 The scenarios described in the embodiments differ from those mentioned above. Since neither of the two scenarios will cause the database memory pressure to exceed the preset pressure limit, the access node can directly write the data packet to be processed into the database memory. In this case, the database can respond to the data packet in a timely manner.

[0056] The various scenarios mentioned in the above embodiments regarding whether or not to control the amount of data written to the database memory can also be combined with... Figure 2 understand.

[0057] In practice, optionally, the access request generated by the UE can be represented as at least one SQL statement based on Structured Query Language (SQL), and one SQL statement can be divided into at least one data packet. Meanwhile, considering that different users have different SQL statement writing habits, the number of data packets divided from one SQL statement can also be different. The data packet to be processed in the various embodiments provided by this invention can be any one of the at least one data packet obtained from the division. Optionally, the size of any data packet divided according to the MySQL protocol is no greater than 16MB. Furthermore, the at least one data packet can be sent to the database buffer in multiple batches, and the at least one data packet can also be stored in the buffer in a first-in-first-out queue.

[0058] After the data packets to be processed in the access node's control queue are stored in the buffer, the access node can continue to execute. Figure 3 The steps in the illustrated embodiment are used to further process other data packets in the queue.

[0059] Figure 3 A flowchart of another access request processing method provided in an embodiment of the present invention. Figure 3 The steps in the illustrated embodiment can be performed as follows: Figure 1 Step S103 in the illustrated embodiment is executed afterward. Figure 3 As shown, the method may include the following steps:

[0060] S201, in the queue, determine the size of the adjacent data packets corresponding to the data packet to be processed.

[0061] S202, determine whether to read adjacent data packets from the buffer based on the size of the adjacent data packets.

[0062] After the data packet to be processed continues to be stored in the buffer, the access node can retrieve the data packet adjacent to the data packet to be processed in the queue. For the sake of brevity in the following description, the data packet adjacent to the data packet to be processed can be referred to as the adjacent data packet corresponding to the data packet to be processed.

[0063] The access node can further determine the size of the adjacent data packet. Similar to the size of the data packet to be processed, for the size of adjacent data packets, one option is for the access node to first read the adjacent data packet into memory and parse it, then determine the size of the adjacent data packet based on the parsing result. Furthermore, after parsing is complete, the vector data packet can be deleted from memory, that is, the memory space occupied for data packet parsing is released.

[0064] Then, the access node can determine whether to read the adjacent data packet from the buffer and write it into the database memory based on the size of the adjacent data packet, that is, whether to continue to control the amount of data written to the database memory.

[0065] One scenario is that if the size of an adjacent data packet is less than a preset threshold, it indicates that the amount of data in the adjacent data packet is small, and the access node can directly write this adjacent data packet into the database's memory. Optionally, this embodiment and... Figure 1 The preset threshold mentioned in the illustrated embodiment can be 16MB.

[0066] Another scenario is that if the size of the adjacent data packet is not less than the preset threshold, it indicates that the data volume of the adjacent data packet is also large. Since the data packet to be processed is also stored in the buffer at this time, the access node can determine again whether the data packet to be processed can be written into the database memory before further processing the adjacent data packets.

[0067] In this embodiment, the access node can continue to process other small data packets in the queue normally after pausing the processing of large-volume pending data packets, while keeping the large-volume pending data packets stored in the buffer. This method ensures that pausing the processing of large-volume pending data packets in the queue does not affect the normal processing of adjacent small-volume data packets in the queue, thus preventing data packet processing blockage.

[0068] Optionally, since the access request corresponding to the data packet in the various embodiments provided by the present invention can specifically be represented as an SQL statement, and this SQL statement can be transmitted using the MySQL protocol based on the TCP protocol, the data packets segmented from the SQL statement can also be transmitted using the MySQL protocol. Furthermore, since the size of the data packet to be processed transmitted using the MySQL protocol is included in the header data, another optional method for determining the size of the data packet to be processed is for the access node to read the header data of the data packet to be processed into the database memory, and obtain the size of the data packet to be processed by parsing the header data.

[0069] Similarly, another alternative method for determining the size of adjacent data packets is for the access node to read the header data of adjacent data packets into the database memory and obtain the size of the adjacent data packets by parsing the header data. Optionally, the size of the header data can be 4 bytes.

[0070] In this embodiment, reading smaller header data into memory to determine the data packet size reduces the database memory footprint and allows for faster determination of the data packet size. This rapid determination of the data packet size enables the access node to more quickly decide whether to control the amount of data written to the database memory.

[0071] Optionally, the process of determining the data packet size mentioned in the various embodiments of the present invention can be handled by any processing thread in the processing thread pool associated with the access node, i.e., the handler thread, and the access node can directly obtain the data packet size determined by the processing thread.

[0072] Specifically, the access node can first obtain the status information of the buffer, which reflects whether the buffer is not empty, that is, whether the buffer contains unprocessed data packets. If the buffer status information indicates that the buffer contains data packets to be processed, the access node can generate a pending event corresponding to the pending data packet. A processing thread is then used to respond to the pending event, thereby ultimately obtaining the size of the data packet.

[0073] Optionally, the access node can periodically obtain the status information of the buffer. The processing thread can also monitor in real time whether there are any pending events.

[0074] It should be noted that the database can be deployed on physical devices. The database can include at least one access node. Any access node can be hardware or software. Different access nodes can also be associated with corresponding buffers, memory and processing thread pools. The buffers, memory and processing thread pools associated with each access node are essentially physical resources provided by the physical device.

[0075] In addition, for data packets transmitted using the MySQL protocol, once the UE has sent at least one data packet corresponding to an access request, and before the UE has received a response to this access request, the UE will not send another access request to the database. That is, as long as the database buffer still stores unprocessed data packets, the UE will not generate and send new data packets to the access node.

[0076] Based on the above characteristics, if the access node continues to store pending data packets in the buffer, it indicates that the current database memory pressure is high. Since the MySQL protocol is based on TCP, the UE can use the mechanisms provided by TCP to know whether there are still unprocessed data packets in the database buffer, thus informing the UE that the database memory pressure is high. Furthermore, once the UE knows that there are still unprocessed data packets in the database buffer, it can pause generating other access requests. This prevents data packets corresponding to other access requests from being sent to the access node, and the access node will not need to read data packets into memory, thereby indirectly reducing the database memory pressure.

[0077] In this embodiment, by using the TCP-based MySQL protocol, the UE can be aware of excessive database memory pressure, thereby slowing down or stopping the generation and transmission of data packets, which is to achieve network backpressure.

[0078] The process of controlling the amount of data written to the database memory mentioned in the above embodiments can also be considered as the process of executing a flow control algorithm on the data packets to be processed. Optionally, different flow control algorithms can correspond to different reference write requirements. Optionally, the reference write requirements may include any one of the following: the total amount of data written to the database within a preset time period is less than a preset quantity, the token bucket corresponding to the database is not empty, and the write cycle of the data packet to be processed has arrived. The preset time period includes taking the generation time of the event to be processed as the start time and a target duration as the end time; the length between the start time and the end time constitutes the preset duration.

[0079] When the current actual write situation of the database is the total amount of data written within the preset time period, and the reference write requirement is that the total amount of data written is less than the preset number, and when the size of the data packet to be processed is greater than the preset threshold, the process of the access node executing the flow control algorithm can be specifically as follows:

[0080] If the total amount of data written to the database within a preset time period exceeds the preset quantity, that is, if the actual writing situation of the database does not meet the current reference writing requirements of the database, then writing a large amount of data packets to be processed directly into memory may cause the database memory pressure to exceed the preset pressure limit. In this case, the access node can control the data packets to be processed to continue to be stored in the buffer, that is, control the amount of data written to the database memory.

[0081] Furthermore, since the access node uses the total amount of data written to the database memory within a preset time period as a reference during the execution of the above flow control algorithm, the above flow control algorithm can also be considered as a counter algorithm or a sliding window algorithm.

[0082] When the current actual write activity of the database is the number of tokens used by the database within a preset time period, and the specific write requirement is that the token bucket corresponding to the database is not empty, and when the size of the data packet to be processed is greater than a preset threshold, the access node can control the amount of data written to the database memory by executing the token bucket algorithm. The specific execution process of the token bucket algorithm is as follows:

[0083] If the number of tokens used by the database within a preset time period is the same as the total number of tokens in the token bucket, meaning the access node determines that the token bucket corresponding to the database is empty, and the current actual write volume of the database does not meet the current reference write requirements, then directly writing this large amount of pending data packets into memory may cause the database's memory pressure to exceed the preset pressure limit. In this case, the access node can control the pending data packets to continue to be stored in the buffer, that is, control the amount of data written to the database memory.

[0084] Optionally, the token bucket used in this embodiment has target parameters, including the target capacity of the token bucket and / or the target generation rate of tokens.

[0085] When the current actual write status of the database includes the generation time of the event to be processed, the reference write requirement is specifically when the write cycle of the data packet to be processed is reached, and when the size of the data packet to be processed is greater than a preset threshold, the process of the access node executing the flow control algorithm can be as follows:

[0086] If the generation time of the event to be processed has not reached the write cycle of the data packet to be processed, that is, the current actual write situation of the database does not meet the current reference write requirements of the database, then directly writing the large data packet to be processed into memory may cause the database memory pressure to exceed the preset pressure limit. In this case, the access node can control the data packet to be processed to continue to be stored in the buffer, that is, control the amount of data written to the database memory.

[0087] Furthermore, since the access node uses the writing cycle of the data packets to be processed as a reference during the execution of the above flow control algorithm, that is, it controls the amount of data written by limiting the writing speed of the data packets, this flow control algorithm can also be called the leaky bucket flow limiting algorithm.

[0088] In this embodiment, the access node can use different flow control algorithms based on different reference write requirements to better control the flow of the data packets to be processed, thereby improving the success rate of database access and ensuring the user experience.

[0089] Furthermore, the contents not described in detail in this embodiment and the technical effects that can be achieved can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0090] Optionally, the target parameters of the token bucket algorithm mentioned in the above embodiments can also be obtained in the following manner. Specifically, the access node can first obtain the initial parameters and initial adjustment step size of the token bucket. Then, based on at least one of the following: the memory usage status of the database, the number of tokens in the token bucket within a preset time period, and the token waiting time, the parameters of the token bucket are iteratively adjusted to obtain the target parameters of the token bucket.

[0091] The database's memory usage status reflects its memory pressure. Optionally, the memory usage status can specifically include any of the following: the database's current remaining memory, the minimum remaining memory within a preset time period, and whether garbage collection has occurred. The number of tokens in the token bucket within the preset time period reflects whether the number of tokens can support the data volume of the current data packet. The token waiting time reflects whether the token generation rate meets the current data packet sending rate.

[0092] Optionally, the initial parameter can be 10 times the historical parameter of the token bucket. Optionally, the initial parameter can also be a preset value. This preset value is usually a large value, such as 1000. Optionally, the initial adjustment step size can be half of the initial parameter.

[0093] For iterative adjustments to the token bucket parameters, optionally, the access node can set a preset number of iterations, meaning that once the preset number of iterations is reached, the adjustment of the token bucket parameters immediately stops. Optionally, after one round of parameter adjustment, the access node can set half of the token bucket parameters obtained in this round as the adjustment step size for the next round, until the adjustment step size is less than a preset threshold, at which point the adjustment of the token bucket parameters stops. Optionally, the preset threshold can be 1 or 2.

[0094] In this embodiment, the access node can automatically iteratively adjust the token bucket parameters to obtain the token parameters. Furthermore, in practice, considering the different database memory specifications and the different SQL statement writing habits of different users, the amount of data contained in the SQL statements written by different users can also vary. Therefore, the access node uses the above iterative adjustment method to obtain algorithm parameters that are adapted to memory specifications, statement writing habits, etc. Compared to manually setting a preset memory pressure limit for each SQL statement, this solution can also reduce labor costs by automatically adjusting the token bucket parameters.

[0095] The automatic iteration process of the token bucket parameters is described in detail below. Specifically, the access node can first determine whether the token waiting time is less than the preset timeout, whether the current remaining memory is less than the preset capacity, and whether the number of tokens in the token bucket is less than the preset number.

[0096] If the token waiting time is not less than the preset timeout, the current remaining memory is not less than the preset capacity, and the number of tokens in the token bucket is not greater than the preset number, it indicates that the token generation speed is slow, the current database memory utilization is low, and the number of currently available tokens is insufficient. In this case, the access node can increase the token bucket parameters according to the adjustment step size corresponding to the current iteration round, that is, increase the token generation speed and / or token bucket capacity, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset number of iterations is reached.

[0097] The token waiting time can be the product of the average token waiting time and the number of connections. The number of connections refers to the number of UEs that have established a communication channel with the access node, and the average token waiting time is the average of the token waiting times of the individual UEs. The preset timeout can be a pre-set maximum token waiting time, which is the maximum waiting time for data packets, meaning that data packets must be processed after waiting for the preset timeout. Optionally, the preset timeout can be 10 minutes, 30 minutes, etc. Optionally, the preset capacity can be half of the total database memory. Optionally, the preset quantity can be 10% of the total token quantity.

[0098] If the token waiting time is less than the preset time, the current remaining memory is less than the preset capacity, and the number of tokens in the token bucket is greater than the preset number, it indicates that the database is under high memory pressure. Optionally, the access node can further obtain the total amount of data written to the database within the preset time. This total amount of data written can reflect the size of the data packets written in a recent period.

[0099] If the total amount of data written to the database within a preset time period is greater than both the preset quantity and the minimum remaining memory, it indicates that the amount of data written within the preset time period is large and the memory of the database with unchanged specifications is insufficient to support the writing of a large amount of data. In this case, the access node can reduce the token generation speed and / or token bucket capacity of the token bucket according to the adjustment step size corresponding to the current iteration round, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset number of iterations is reached.

[0100] If the total amount of data written to the database within a preset time period exceeds a preset limit and the database memory has undergone garbage collection, it indicates that the amount of data written within the preset time period is large and the memory with unchanged specifications cannot support the writing of a large number of data packets. In this case, the access node can adjust the step size according to the current iteration round, reducing the token generation speed and / or token bucket capacity. Garbage collection includes full garbage collection (FGC) and / or out-of-memory (OOM).

[0101] If the total amount of data written to the database within the preset time period is less than the preset number and the database memory has not undergone garbage collection, it indicates that the amount of data written within the preset time period is moderate and the memory can support the writing of data packets. In this case, the access node can keep the current token bucket parameters unchanged, that is, there is no need to adjust the token bucket parameters.

[0102] If the total amount of data written is less than the preset amount and the total amount of data written is less than the minimum remaining memory, it indicates that the amount of data written within the preset time period is moderate and the database has a lot of remaining memory. In this case, the access node can keep the current token bucket parameters unchanged, that is, there is no need to adjust the token bucket parameters.

[0103] The iterative adjustment process of the token bucket parameters mentioned above can also be combined with Figure 4 understand.

[0104] In this embodiment, the access node can accurately determine whether to increase or decrease the token generation speed and / or token bucket capacity based on the token waiting time, current remaining memory, number of tokens in the token bucket, total amount of data written, minimum remaining memory of the database within a preset time, and whether the database memory has undergone garbage collection, so as to better adjust the token bucket parameters.

[0105] Furthermore, the contents not described in detail in this embodiment and the technical effects that can be achieved can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0106] Optionally, the access node mentioned in the above embodiments is also a database node in the database. Optionally, as... Figure 5 As shown, the database can be deployed on a physical device, and the physical device can allocate corresponding buffers and memory for this access node.

[0107] The working process of the access node in the database may include: the access node can first determine the size of the data packets to be processed in the buffer.

[0108] Then, based on the size of the data packet to be processed, the access node can further determine whether to read the data packet from the buffer. That is, the size of the data packet to be processed is used as a basis to determine whether to control the amount of data written to the database memory.

[0109] Optionally, if the size of the data packet to be processed is not less than a preset threshold, it indicates that the amount of data in the data packet to be processed is large. Furthermore, if the current actual write situation of the database does not meet the current reference write requirements of the database, it indicates that directly writing this large data packet to memory may cause the database memory pressure to exceed the preset pressure limit. In this case, the access node can control the data packet to be processed to continue to be stored in the buffer, that is, control the amount of data written to the database memory.

[0110] Optionally, the data packet to be processed may correspond to an access request to the database generated by the UE's user, and the access request may include the size information of the data packet to be processed. Furthermore, the response to the access request is completed upon completion of the data packet processing.

[0111] Optionally, to determine the size of the data packet, the access node can read the data packet to be processed into the database memory and parse it. The size of the data packet to be processed can be determined based on the parsing result. Furthermore, to prevent the data packet from consuming too much memory, it can be deleted from memory after parsing, thus releasing the memory space occupied for data packet parsing.

[0112] In this embodiment, the access node can first determine the size of the data packet to be processed. This data packet, which is network data, can be sent from the user equipment's buffer and written by the access node into the database access request buffer. Then, the access node determines whether to read the data packet from the buffer based on its size, which is equivalent to determining whether to control the amount of data written to the database memory. When the size of the data packet to be processed is not less than a preset threshold, and the current actual write volume of the database does not meet the current reference write requirements, it indicates that the database memory is currently strained. In this case, the access node can control the amount of data written to the database memory, that is, it can refrain from reading the data packet from the buffer, allowing the data packet to continue to be stored in the database buffer.

[0113] The above method provides a way to control the amount of data written to memory. The access node can control the continued storage of large data packets in the buffer, thereby reducing the data write speed and volume to the database memory and thus reducing memory pressure. Compared to reducing memory pressure by discarding data packets, this method avoids errors indicating database access failures caused by packet dropping. Therefore, the flow control method provided by this approach is more reasonable, as it can reduce database memory pressure while improving the success rate of database access and ensuring a better user experience.

[0114] Furthermore, for any content not described in detail in this embodiment and the technical effects that can be achieved, please refer to the relevant descriptions in the above embodiments of the method, which will not be repeated here.

[0115] In practice, a database access node can include at least one. Figure 6 This is a schematic diagram of another physical device provided in an embodiment of the present invention. (See diagram below.) Figure 6 As shown, a physical device with a database deployed can allocate buffers and memory for any access node in the database.

[0116] The working process of any access node can be described as follows: first, determine the size of the data packets to be processed in the buffer.

[0117] Then, based on the size of the data packet to be processed, any access node can further determine whether to write the data packet to be processed from the database buffer to the database memory. That is, the size of the data packet to be processed is used as a basis to determine whether to control the amount of data written to the database memory.

[0118] Optionally, if the size of the data packet to be processed is not less than a preset threshold, it indicates that the amount of data in the data packet to be processed is large. Furthermore, if the current actual write situation of the database does not meet the current reference write requirements of the database, it indicates that directly writing this large data packet to memory may cause the database memory pressure to exceed the preset pressure limit. In this case, any access node can control the data packet to be processed to continue to be stored in the buffer, that is, control the amount of data written to the database memory.

[0119] Optionally, the data packet to be processed may correspond to an access request to the database generated by the UE's user. And once the data packet is processed, the response to the access request is complete.

[0120] Optionally, to determine the size of the data packet, any access node can read the data packet to be processed into the database memory and parse it. The size of the data packet to be processed can be determined based on the parsing result. Furthermore, to prevent the data packet from consuming too much memory, it can be deleted from memory after parsing, thus releasing the memory space occupied for data packet parsing.

[0121] In this embodiment, the access node can first determine the size of the data packet to be processed. This data packet, which is network data, can be sent from the user equipment's buffer and written into the database access request buffer by the database node. Then, the access node can determine whether to control the amount of data written to the database memory based on the size of the data packet. Specifically, when the size of the data packet to be processed is not less than a preset threshold, and the current actual write volume of the database does not meet the current reference write requirements, it indicates that the database memory is currently tight. In this case, the access node can control the amount of data written to the database memory, that is, it will not read the data packet to be processed from the buffer, and will continue to store the data packet to be processed in the database buffer.

[0122] The above method provides a way to control the amount of data written to memory. Any access node can control the continued storage of large data packets in the buffer, thereby reducing the data write speed and volume to the database memory and thus reducing memory pressure. Compared to reducing memory pressure by discarding data packets, this method avoids errors indicating database access failures caused by packet dropping. Therefore, the flow control method provided by this approach is more reasonable, as it can reduce database memory pressure while also improving the success rate of database access and ensuring a better user experience.

[0123] Furthermore, for any content not described in detail in this embodiment and the technical effects that can be achieved, please refer to the relevant descriptions in the above embodiments of the method, which will not be repeated here.

[0124] The following will describe in detail one or more embodiments of the access request processing apparatus of the present invention. Those skilled in the art will understand that the search apparatus can be configured using commercially available hardware components through the steps taught in this solution.

[0125] Figure 7 This is a schematic diagram of the structure of an access request processing device provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the device includes:

[0126] The data packet size determination module 11 is used to determine the size of the data packet to be processed in the buffer of the database access request. The data packet to be processed is contained in the database access request, and the access request contains the size information of the data packet to be processed.

[0127] The memory write determination module 12 is used to determine whether to read the data packet to be processed from the buffer based on the size information of the data packet to be processed.

[0128] The control module 13 is configured to control the data packet to continue to be stored in the buffer if the size of the data packet to be processed is not less than a preset threshold and the current actual write status of the database does not meet the current reference write requirements of the database.

[0129] The current reference write requirements for the database include any of the following: the total amount of data written to the database within a preset time period is less than a preset number, the preset time period includes a preset duration with the generation time of the event to be processed as the starting time; the token bucket corresponding to the database is not empty; the write cycle of the data packet to be processed has been reached.

[0130] Optionally, the device further includes a writing module 14, which is used to write the data packet to be processed read from the buffer into the memory of the database if the size of the data packet to be processed is less than the preset threshold.

[0131] Optionally, the access request includes at least one data packet stored in the buffer in the form of a queue, and the data packet to be processed is any one of the at least one data packet.

[0132] The data packet size determination module 11 is used to determine the size of the adjacent data packets corresponding to the data packet to be processed in the queue.

[0133] The memory write determination module 12 is used to determine whether to read the adjacent data packet from the buffer based on the size of the adjacent data packet.

[0134] Optionally, the writing module 14 is configured to write the data packet to be processed in the buffer into the memory of the database if the size of the data packet to be processed is not less than the preset threshold and the current actual writing situation of the database meets the current reference writing requirements of the database.

[0135] Optionally, the data packet size determination module 11 is used to obtain the status information of the buffer; if the status information of the buffer reflects that the buffer contains the data packet to be processed, then generate a pending event corresponding to the data packet to be processed; use a processing thread to respond to the pending event to read the header data of the data packet to be processed into the memory; parse the header data to obtain the size of the data packet to be processed.

[0136] Optionally, the current reference write requirement for the database includes that the token bucket corresponding to the database is not empty. Based on this, the device further includes an adjustment module 15, used to iteratively adjust the parameters of the token bucket according to at least one of the memory usage status of the database, the number of tokens in the token bucket within the preset time period, and the token waiting time, so as to obtain the target parameters of the token bucket.

[0137] Optionally, the memory usage status includes the current remaining memory of the database; the parameters of the token bucket include the token generation rate and / or the token bucket capacity.

[0138] The adjustment module 15 is used to increase the token generation speed and / or token bucket capacity of the token bucket according to the adjustment step size corresponding to the current iteration round if the waiting time of the token is not less than the preset timeout time, the current remaining memory is not less than the preset capacity, and the number of tokens in the token bucket is not greater than the preset number, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset number of iterations is reached.

[0139] Optionally, the memory usage status includes the minimum remaining memory of the database within the preset time period.

[0140] The adjustment module 15 is used to obtain the minimum remaining memory if the waiting time of the token is less than the preset timeout duration, the current remaining memory is less than the preset capacity, and the number of tokens in the token bucket is greater than the preset number; if the total amount of data written is greater than both the preset number and the minimum remaining memory, the token generation speed and / or the token bucket capacity of the token bucket are reduced according to the adjustment step size corresponding to the current iteration round, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset number of iterations is reached.

[0141] Optionally, the memory usage status includes whether the database memory has undergone garbage collection.

[0142] The adjustment module 15 is configured to reduce the token generation speed and / or token bucket capacity of the token bucket according to the adjustment step size corresponding to the current iteration round if the waiting time of the token is less than the preset timeout duration, the current remaining memory is less than the preset capacity, the number of tokens in the token bucket is greater than the preset number, the total amount of data written is greater than the preset number, and the memory of the database has undergone garbage collection, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset number of iterations is reached.

[0143] Figure 7 The device shown can perform Figures 1-4 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figures 1-4 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figures 1-4 The descriptions in the illustrated embodiments will not be repeated here.

[0144] In one possible design, the access request processing method provided in the above embodiments can be applied to an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 21 and a memory 22. The memory 22 is used to store data that supports the electronic device in performing the above-described actions. Figures 1-4 The program for the access request processing method provided in the illustrated embodiment is such that the processor 21 is configured to execute the program stored in the memory 22.

[0145] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 21, can perform the following steps:

[0146] Determine the size of the data packet to be processed in the buffer of the database access request, wherein the data packet to be processed is contained in the database access request and the access request contains the size information of the data packet to be processed;

[0147] Based on the size information of the data packet to be processed, determine whether to read the data packet to be processed from the buffer;

[0148] If the size of the data packet to be processed is not less than a preset threshold, and the current actual write status of the database does not meet the current reference write requirements of the database, then the data packet to be processed is controlled to continue to be stored in the buffer.

[0149] The structure of the electronic device may also include a communication interface 23 for the electronic device to communicate with other devices or communication networks.

[0150] Furthermore, embodiments of the present invention provide a non-transitory machine-readable storage medium for storing computer software instructions used in the aforementioned electronic device, which includes instructions for executing the above-mentioned... Figures 1-4 The procedures involved in the access request handling method shown.

[0151] In addition, embodiments of the present invention provide a computer program product. This computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the processor is able to perform the above-described functions. Figures 1-4 The steps or functions of the access request processing method shown.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An access request processing method, characterized in that, include: Determine the size of the data packet to be processed in the buffer of the database access request, wherein the data packet to be processed is contained in the database access request and the access request contains the size information of the data packet to be processed; Based on the size information of the data packet to be processed, determine whether to read the data packet to be processed from the buffer; If the size of the data packet to be processed is not less than a preset threshold, and the current actual write status of the database does not meet the current reference write requirements of the database, then the data packet to be processed is controlled to continue to be stored in the buffer.

2. The method according to claim 1, characterized in that, The method further includes: If the size of the data packet to be processed is less than the preset threshold, the data packet to be processed read from the buffer will be written into the memory of the database.

3. The method according to claim 1, characterized in that, The access request includes at least one data packet stored in the buffer in the form of a queue, and the data packet to be processed is any one of the at least one data packets; After controlling the data packet to be processed to continue to be stored in the buffer, the method further includes: In the queue, the size of the adjacent data packets corresponding to the data packet to be processed is determined; Based on the size of the adjacent data packets, determine whether to read the adjacent data packets from the buffer.

4. The method according to claim 1, characterized in that, The method further includes: If the size of the data packet to be processed is not less than the preset threshold, and the current actual write status of the database meets the current reference write requirements of the database, then the data packet to be processed read from the buffer will be written into the memory of the database.

5. The method according to claim 1, characterized in that, The data packet supports the Structured Query Language protocol, and determining the size of the data packet to be processed in the database buffer includes: Obtain the status information of the buffer; If the status information of the buffer reflects that the buffer contains the data packet to be processed, then a processing event corresponding to the data packet to be processed is generated; The processing thread responds to the event to be processed, so as to read the header data of the data packet to be processed into the memory; The header data is parsed to obtain the size of the data packet to be processed.

6. The method according to claim 5, characterized in that, The current reference write requirements for the database include any of the following: The total amount of data written to the database within a preset time period is less than a preset number, and the preset time period includes a preset duration with the generation time of the event to be processed as the starting time. The token bucket corresponding to the database is not empty; The write cycle of the data packet to be processed has been reached.

7. The method according to claim 6, characterized in that, The current reference write requirement for the database includes that the token bucket corresponding to the database is not empty; the method further includes: Based on at least one of the following: the memory usage status of the database, the number of tokens in the token bucket within the preset time period, and the token waiting time, the parameters of the token bucket are iteratively adjusted to obtain the target parameters of the token bucket.

8. The method according to claim 7, characterized in that, The memory usage status includes the current remaining memory of the database; the parameters of the token bucket include the token generation speed and / or the token bucket capacity. The step of iteratively adjusting the parameters of the token bucket based on at least one of the following: the memory usage status of the database, the number of tokens in the token bucket within the preset time period, and the token waiting time, includes: If the waiting time for the token is not less than the preset timeout duration, the current remaining memory is not less than the preset capacity, and the number of tokens in the token bucket is not greater than the preset number, then the token generation speed and / or the token bucket capacity are increased according to the adjustment step size corresponding to the current iteration round, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset number of iterations is reached.

9. The method according to claim 8, characterized in that, The memory usage status includes the minimum remaining memory of the database within the preset time period; The step of iteratively adjusting the parameters of the token bucket based on at least one of the following: the memory usage status of the database, the number of tokens in the token bucket within the preset time period, and the token waiting time, includes: If the waiting time of the token is less than the preset timeout time, the current remaining memory is less than the preset capacity, and the number of tokens in the token bucket is greater than the preset number, then the minimum remaining memory is obtained. If the total amount of data written is greater than both the preset quantity and the minimum remaining memory, then the token generation speed and / or token bucket capacity of the token bucket will be reduced according to the adjustment step size corresponding to the current iteration round, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset iteration number is reached.

10. The method according to claim 8, characterized in that, The memory usage status includes whether the database memory has undergone garbage collection; The step of iteratively adjusting the parameters of the token bucket based on at least one of the following: the memory usage status of the database, the number of tokens in the token bucket within the preset time period, and the token waiting time, includes: If the token waiting time is less than the preset timeout time, the current remaining memory is less than the preset capacity, the number of tokens in the token bucket is greater than the preset number, the total amount of data written is greater than the preset number, and the database memory has undergone garbage collection, then the token generation speed and / or token bucket capacity of the token bucket will be reduced according to the adjustment step size corresponding to the current iteration round, until the adjustment step size corresponding to the current iteration round is less than the preset threshold or the preset number of iterations is reached.

11. A database node, characterized in that, The physical device where the database is deployed allocates buffers and memory for the database nodes; The database node is used to determine the size of the data packet to be processed in the buffer. The data packet to be processed is contained in the database access request, and the access request contains the size information of the data packet to be processed. Based on the size information of the data packet to be processed, determine whether to read the data packet to be processed from the buffer; If the size of the data packet to be processed is not less than a preset threshold, and the current actual write status of the database does not meet the current reference write requirements of the database, then the data packet to be processed is controlled to continue to be stored in the buffer.

12. A database, characterized in that, Deployed in a physical device, the database includes at least one access node; the physical device allocates buffers and memory for any access node in the database; The access node is used to determine the size of the data packet to be processed in the buffer, the data packet to be processed is contained in the access request of the database, and the access request contains the size information of the data packet to be processed; Based on the size information of the data packet to be processed, determine whether to read the data packet to be processed from the buffer; If the size of the data packet to be processed is not less than a preset threshold, and the current actual write status of the database does not meet the current reference write requirements of the database, then the data packet to be processed is controlled to continue to be stored in the buffer.

13. An electronic device, characterized in that, include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the access request processing method as described in any one of claims 1 to 10.

14. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the access request processing method as described in any one of claims 1 to 10.

15. A computer program product, characterized in that, The computer program product includes a computer program or instructions that enable the computer program or instructions to perform the steps of the access request processing method according to any one of claims 1 to 10.

Images