Data traversal method, database system, electronic device, and computer storage medium

Abstract

Embodiments of the present application provide a data traversal method, a database system, an electronic device and a computer storage medium. The data traversal method comprises: determining a current cursor interval corresponding to to-be-traversed data, and traversing data in the current cursor interval based on a cursor to determine a total amount of data that has been traversed; calculating an average data amount of the current cursor interval in a predetermined interval unit based on the total amount of data and the predetermined interval unit, wherein the interval unit is a data range unit for measuring the to-be-traversed data; and determining a number of interval units corresponding to a next cursor interval based on the average data amount corresponding to the interval unit, to determine the next cursor interval.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data traversal method, a database system, an electronic device, and a computer storage medium. Background Technology

[0002] Traversing data in a database to retrieve data that meets query criteria is a common database practice. However, when performing data traversal, since the size of the data to be traversed is unknown—that is, when performing data traversal operations on an unknown scale—certain methods are employed to control the amount of data traversed each time.

[0003] For example, one approach is a pagination traversal mode, which involves paginating the data in the database and then sequentially traversing it according to the page number and the amount of data per page to control the range of data traversed each time. The biggest drawback of this traversal method is that as the page number increases, the number of random 1 / 0 operations in the database increases, reducing the database query performance. Another approach is mainly suitable for scenarios where the primary key is evenly distributed. It can use the primary key to evenly divide the data, but the disadvantage of this method is that the primary key of the data must be evenly distributed, resulting in poor versatility.

[0004] In view of this, the technical problem that the existing technology urgently needs to solve is to provide an alternative data traversal scheme that has high versatility while minimizing the reduction in database query performance. Summary of the Invention

[0005] In view of this, embodiments of this application provide a data traversal scheme to at least partially solve the above-mentioned problems.

[0006] According to a first aspect of the embodiments of this application, a data traversal method for a database is provided, comprising: determining the current cursor interval corresponding to the data to be traversed, and traversing the data in the current cursor interval based on the cursor to determine the total amount of data to be traversed; calculating the average amount of data in the current cursor interval within the interval unit based on the total amount of data and a predetermined interval unit, wherein the interval unit is a unit for measuring the data range of the data to be traversed; and determining the number of interval units corresponding to the next cursor interval based on the average amount of data corresponding to the interval unit, thereby determining the next cursor interval.

[0007] According to a second aspect of the embodiments of this application, a data filtering method is provided, comprising: obtaining a first filtering condition and a second filtering condition, wherein the first filtering condition is a condition for filtering by a database index, and the second filtering condition is a condition that cannot be filtered by the database index; performing a query operation corresponding to the first filtering condition based on the database index to obtain data to be traversed; obtaining the current cursor interval corresponding to the data to be traversed, and using the second filtering condition as a query condition, traversing the data in the current cursor interval based on the cursor to obtain data matching the second filtering condition, and determining the total amount of data traversed; calculating the average amount of data in the current cursor interval within the interval unit based on the total amount of data and a predetermined interval unit, wherein the interval unit is a data range unit for measuring the data to be traversed; determining the number of interval units corresponding to the next cursor interval based on the average amount of data corresponding to the interval unit to determine the next cursor interval, and continuing to execute the steps of using the second filtering condition as a query condition, traversing the data in the current cursor interval based on the cursor to obtain data matching the second filtering condition, and determining the total amount of data traversed.

[0008] According to a third aspect of the embodiments of this application, a database system is provided, comprising: a data interval traversal module, configured to determine the current cursor interval corresponding to the data to be traversed, and determine the total amount of data to be traversed based on the data in the current cursor interval traversed by the cursor; and a data interval stable partitioning module, configured to calculate the average amount of data in the current cursor interval within the interval unit based on the total amount of data and a predetermined interval unit, wherein the interval unit is a unit for measuring the data range of the data to be traversed, and to determine the number of interval units corresponding to the next cursor interval based on the average amount of data corresponding to the interval unit, thereby determining the next cursor interval.

[0009] According to a fourth aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described above.

[0010] According to a fifth aspect of the present application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method described above.

[0011] According to the data traversal scheme provided in the embodiments of this application, the current cursor interval corresponding to the data to be traversed is determined, and the total amount of data to be traversed is determined based on the data traversed in the current cursor interval. Based on the total amount of data and a predetermined interval unit, the average amount of data in the current cursor interval within the interval unit is calculated, wherein the interval unit is a unit for measuring the data range of the data to be traversed. Based on the average amount of data corresponding to the interval unit, the number of interval units corresponding to the next cursor interval is determined to determine the next cursor interval. This method does not require the creation of new indexes, the traversal process does not rely on uniformly distributed primary keys, has good versatility, and can reduce the number of I / O operations during the traversal process, thus ensuring the query performance of the database. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 A schematic diagram of an exemplary system for which the data traversal method of the embodiments of this application is applicable;

[0014] Figure 2 This is a flowchart illustrating the steps of a data traversal method according to an embodiment of this application;

[0015] Figure 3 This is a flowchart illustrating the steps of a method for traversing data within a cursor interval according to an embodiment of this application.

[0016] Figure 4 This is a flowchart illustrating the steps of a data filtering method according to Embodiment 3 of this application;

[0017] Figure 5 This is a structural block diagram of a database system according to Embodiment 4 of this application;

[0018] Figure 6 This is a schematic diagram of the structure of an electronic device according to Embodiment 5 of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0020] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0021] Figure 1 An exemplary system for a data traversal method applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system may include a cloud server 102, a communication network 104, and / or one or more user devices 106. Figure 1 The example in the text shows multiple user devices.

[0022] The cloud server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the cloud server 102 can perform any suitable function. For example, in some embodiments, the cloud server 102 can act as a database for storing and traversing data.

[0023] In some embodiments, communication network 104 may be any suitable combination of one or more wired and / or wireless networks. For example, communication network 104 may include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. User equipment 106 may be connected to communication network 104 via one or more communication links (e.g., communication link 112), and communication network 104 may be linked to cloud server 102 via one or more communication links (e.g., communication link 114). Communication links may be any communication link suitable for transmitting data between user equipment 106 and cloud server 102, such as network links, dial-up links, wireless links, hardwired links, any other suitable communication links, or any suitable combination of such links.

[0024] User device 106 may include any one or more user devices suitable for setting filtering conditions, receiving data query operations against a database, and displaying queried data. In some embodiments, user device 106 may include any suitable type of device. For example, in some embodiments, user device 106 may include mobile devices, tablet computers, laptop computers, desktop computers, wearable computers, game consoles, media players, vehicle entertainment systems, and / or any other suitable type of user device.

[0025] Based on the above system, this application provides a data traversal method, which will be described below through several embodiments.

[0026] See Figure 2 A flowchart illustrating a database data traversal method is provided, as shown in the figure, which includes:

[0027] S201. Determine the current cursor interval corresponding to the data to be traversed, and traverse the data in the current cursor interval based on the cursor to determine the total amount of data to be traversed.

[0028] The solution provided in this embodiment can be used to traverse incrementally increasing data, which means that the newly generated data each day is not repeated from the data generated in previous periods; that is, all data generated each day is new. Since the amount of data generated each day is uncertain, the order of magnitude of the data being traversed is unknown. Of course, the above is merely an example; traversal solutions for other types of databases are also within the scope of this application.

[0029] In this embodiment, the cursor interval refers to the range of data corresponding to the data to be traversed. Specifically, it can be the range in which the cursor slides when traversing data. For example, the cursor interval can be from the data generated on day N to the data generated on day N+2. Specifically, in order to avoid the data being traversed repeatedly, the cursor interval can be left-open and right-closed or left-closed and right-open.

[0030] A cursor provides the ability to move forward or backward through data within a cursor range, one or more rows at a time. Traversing the data within the current cursor range using a cursor can reduce the number of I / O operations during the traversal process, ensuring that database query performance is minimally affected. For specific methods of data traversal using cursors, please refer to relevant technical resources; they will not be elaborated upon here.

[0031] Optionally, in this embodiment, step S201 may specifically include: performing a pagination operation on the current cursor range according to a preset pagination size to obtain several data pages; traversing several data pages sequentially based on an incrementing cursor until the size of the traversed data page is less than the preset pagination size to end the traversal; and determining the total amount of data traversed based on the traversal results.

[0032] The solution provided in this embodiment traverses each data page based on a cursor until the size of the currently traversed data page is less than the preset data page size. This indicates that the currently traversed data page is the last data page in the current cursor interval, at which point the traversal process can be ended, and the total amount of data traversed can be determined. Compared with the traditional pagination traversal method, which requires deep pagination for each traversed data page, resulting in more random I / O operations with deeper pagination, the solution provided in this embodiment traverses based on a cursor, eliminating the need for deep pagination, greatly reducing the number of random I / O operations and ensuring data query efficiency.

[0033] For example, in this embodiment, see Figure 3 This shows a flowchart of a traversal method, which can be executed for each current cursor range. Figure 3 The flowchart shown is traversed.

[0034] like Figure 3 As shown, step S201 may specifically include:

[0035] S2011. Initialize the preset page size (pagesize), cursor start value (cursor), and duplicated key-value redundant list (duplicatedlist).

[0036] The preset page size is used to determine the number of data pages, and the cursor start value can be the starting storage location of the current cursor range. A duplicate key-value redundancy list is used to store duplicate key-value pairs to avoid repeated traversal.

[0037] S2012. Based on the traversal conditions, start traversing from the cursor's initial value until the data corresponding to the preset page size (pagesize) has been traversed.

[0038] S2013. During the traversal, skip the key values ​​stored in the duplicate key-value redundancy list and do not traverse them.

[0039] Specifically, the key values ​​in the duplicate key-value redundancy list can be used as traversal conditions to eliminate redundant data.

[0040] S2014. Calculate the starting value of the next cursor based on the traversal results.

[0041] Then step S2012 can be executed again until the amount of data being traversed is less than the amount of data corresponding to the preset page size.

[0042] Specifically, the key value corresponding to the last piece of data being traversed can be determined as the starting value of the next cursor, and the cursor starting value can be recorded in the duplicate key value redundancy list during subsequent traversal to avoid repeated traversal.

[0043] When the amount of data being iterated is less than the amount of data corresponding to the preset page size, it means that the data being iterated this time is not enough to be divided into a separate data page, and there is no other data to follow, that is, the traversal is complete.

[0044] S202. Based on the total data volume and the predetermined interval unit, calculate the average data volume of the current cursor interval within the interval unit.

[0045] In this embodiment, the interval unit is a unit for measuring the data range of the data to be traversed. Specifically, the interval unit can be a unit of time, or it can be a data source, etc.; this embodiment does not limit this.

[0046] For example, taking an interval as the unit of time, if the current cursor interval can be the data generated on day N to the data generated on day N+2, then the average amount of data within the interval can be the average amount of data per unit of time "day". Alternatively, it can be the average amount of data per hour within the current cursor interval.

[0047] Optionally, in this embodiment, the interval unit is a unit of time, then step S202 may include: calculating the ratio of the total amount of data to the time range of the current cursor interval, and determining the average amount of data per unit of time.

[0048] S203. Based on the average data volume corresponding to the interval unit, determine the number of interval units corresponding to the next cursor interval, so as to determine the next cursor interval.

[0049] Optionally, in this embodiment, step S203 may include: calculating the number of interval units corresponding to the next cursor interval according to a preset calculation scheme based on the average data volume corresponding to the interval unit and a preset traversal order of magnitude threshold, so as to determine the next cursor interval. The preset calculation scheme includes at least one of the following: performing a proportional calculation on the average data volume and the traversal order of magnitude threshold; performing an integral operation on the average data volume with respect to the interval unit and multiplying it by the traversal order of magnitude threshold; performing a differential operation on the average data volume with respect to the interval unit and multiplying it by the traversal order of magnitude threshold.

[0050] Specifically, the formula for proportionally calculating the average data volume and the threshold of the traversal order of magnitude is as follows:

[0051]

[0052] Where i represents the current cursor interval, and i+1 represents the next cursor interval. The average data volume corresponding to the unit of the current cursor interval. This represents the number of units in the next cursor interval. This is the proportional control coefficient. A pre-set threshold for the number of iterations.

[0053] The aforementioned control scheme can be specifically termed proportional control, meaning the number of units to be picked up is inversely proportional to the average data volume. However, when implementing this proportional control scheme, significant fluctuations in the data volume within the cursor interval may occur, leading to uneven division of the cursor interval. For example, the initially defined cursor interval might be [0, 100], but the next cursor interval determined based on the initially defined interval might become [100, 10000], resulting in uneven division.

[0054] Therefore, in this embodiment, integration can also be used. Specifically, the average data volume is integrated over the interval unit and multiplied by the calculation formula of the traversal order of magnitude threshold:

[0055]

[0056] Where i represents the current cursor interval, and i+1 represents the next cursor interval. The average data volume corresponding to the unit of the current cursor interval. This represents the number of units in the next cursor interval. The integral control coefficient, A pre-set threshold for the order of magnitude of traversal, where x indicates the unit of the interval.

[0057] According to the above formula, the integral term... Also related to average data volume The integral term is inversely proportional to the data size, and integral control can ensure that uneven division does not occur. However, it is possible that a small average data size may lead to an excessively large integral term, resulting in a large data size in the divided cursor intervals.

[0058] Therefore, in this embodiment, differential operations can also be used. The formula for calculating the average data volume by differentiating it from the interval unit and multiplying it by the threshold of the traversal order of magnitude is as follows:

[0059]

[0060] Where i represents the current cursor interval, and i+1 represents the next cursor interval. The average data volume corresponding to the unit of the current cursor interval. This represents the number of units in the next cursor interval. The differential control coefficient, A pre-set threshold for the order of magnitude of traversal, where x indicates the unit of the interval.

[0061] By adding differential control, the rate of change of the vernier interval in the interval unit can be reduced, thus resisting the change in data and ensuring that the change in data remains within the set stable data level.

[0062] It should also be noted that the above three adjustment schemes can be used simultaneously, that is, the formula for calculating the number of interval units corresponding to the next cursor interval can be:

[0063]

[0064] For detailed parameters, please refer to the box quantity section, which will not be repeated here.

[0065] It should be noted that the values ​​of p1, p2, and p3 in the above formula can be adjusted and tested according to the changes in the data volume of large-scale transaction form data, and the value of Ls can be determined by relevant staff based on the database traversal efficiency.

[0066] Since the amount of data received by the database generally does not change abruptly, the solution provided in this embodiment uses the average amount of data within the current cursor interval to determine the number of interval units corresponding to the next cursor interval. That is, through feedback control logic, based on the average amount of data in the current cursor interval's corresponding unit, it aims to ensure that the amount of data in the next cursor interval remains within a certain range. This ensures that the amount of data traversed during the data traversal process is within a certain range. Compared with existing data traversal methods, the solution provided in this embodiment does not require the creation of new indexes, the traversal process does not rely on uniformly distributed primary keys, has better versatility, and can reduce the number of I / O operations during the traversal process, thus ensuring the database query performance. Furthermore, in this embodiment, by setting the above adjustment calculation formula, multiple adjustments can be made to stabilize the amount of data in the cursor interval from high fluctuations, improving traversal efficiency.

[0067] See Figure 4 The figure shows a flowchart of a data filtering method, which includes:

[0068] S401. Obtain a first filtering condition and a second filtering condition, wherein the first filtering condition is a condition for filtering by a database index, and the second filtering condition is a condition that cannot be filtered by the database index.

[0069] In this embodiment, the first filtering condition can be a filtering condition that is supported by default when the form is created, and the second filtering condition is a filtering condition added during the use of the form.

[0070] S402. Perform a query operation corresponding to the first filtering condition based on the database index to obtain the data to be traversed;

[0071] S403. Obtain the current cursor interval corresponding to the data to be traversed, and use the second filtering condition as the query condition. Traverse the data in the current cursor interval based on the cursor to obtain the data that matches the second filtering condition, and determine the total amount of data to be traversed.

[0072] S404. Based on the total amount of data and the predetermined interval unit, calculate the average amount of data in the current cursor interval within the interval unit, wherein the interval unit is a unit for measuring the data range of the data to be traversed.

[0073] S405. Based on the average data volume corresponding to the interval unit, determine the number of interval units corresponding to the next cursor interval, so as to determine the next cursor interval, and continue to execute the steps of using the second filtering condition as the query condition, traversing the data in the current cursor interval based on the cursor, obtaining the data that matches the second filtering condition, and determining the total amount of data traversed.

[0074] For example, the first filter condition could be a filter that allows filtering exported data based on form type, or a filter that allows filtering based on the index used when the form was created. If a user adds a filter based on a combination of form type and form creator, the second filter condition could be the form creator. If a certain type of form data has a large volume, adding a form creator filter field without changing the index would require the database to perform external sorting and traversal operations, leading to a significant decrease in database query performance. Adding new indexes to meet constantly changing user filtering needs results in long database change cycles, high online risks, and increasingly large index usage, leading to high costs and a poor user experience.

[0075] The solution provided in this embodiment first performs a query based on the query operation corresponding to the first filtering condition, and then iterates through the data to be traversed determined by the query through steps S403-S405. It does not require adding an index, has high versatility, and does not increase the number of random I / O operations in the database, thus ensuring the query efficiency of the database.

[0076] See Figure 5 This application provides a schematic diagram of the structure of a database system, as shown in the figure, which includes:

[0077] The data interval traversal module 501 is used to determine the current cursor interval corresponding to the data to be traversed, and to traverse the data in the current cursor interval based on the cursor to determine the total amount of data to be traversed;

[0078] The data interval stable partitioning module 502 is used to calculate the average amount of data in the current cursor interval within the interval unit based on the total amount of data and the predetermined interval unit, wherein the interval unit is a data range unit for measuring the data to be traversed, so as to determine the number of interval units corresponding to the next cursor interval based on the average amount of data corresponding to the interval unit, thereby determining the next cursor interval.

[0079] Optionally, see Figure 5 When the proportional control, integral control, and derivative control in the above embodiments are used, the data interval stabilization division module 502 may include a proportional control module, an integral control module, and a derivative control module, which are used to perform corresponding calculations respectively. The negative feedback indicator in the figure is calculated based on the reciprocal of the average data volume.

[0080] It should be noted that, see Figure 5 When performing a traversal, the first cursor interval can be determined during the initialization process according to the pre-set parameters, and the other cursor intervals after the first cursor interval are determined by the data interval stable partitioning module 502.

[0081] Reference Figure 6 The diagram shows a structural schematic of an electronic device according to Embodiment 5 of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0082] like Figure 6 As shown, the electronic device may include: a processor 602, a communications interface 604, a memory 606, and a communications bus 608.

[0083] in:

[0084] The processor 602, communication interface 604, and memory 606 communicate with each other via communication bus 608.

[0085] Communication interface 604 is used for communication with other electronic devices or servers.

[0086] The processor 602 is used to execute program 610, specifically to execute the relevant steps in the above data traversal method embodiment.

[0087] Specifically, program 610 may include program code that includes computer operation instructions.

[0088] The processor 602 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0089] Memory 606 is used to store program 610. Memory 606 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0090] Specifically, program 610 can be used to cause processor 602 to perform the operation corresponding to the method described in any of the foregoing multiple method embodiments.

[0091] The specific implementation of each step in program 610 can be found in the corresponding steps and units described in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0092] This application also provides a computer storage medium storing a computer program that, when executed by a processor, implements any of the methods described in the above-described multiple method embodiments.

[0093] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described multiple method embodiments.

[0094] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0095] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0096] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0097] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A data traversal method of a database, comprising: determining a current cursor interval corresponding to data to be traversed, and determining a total amount of data traversed based on traversing the data in the current cursor interval by a cursor; calculating an average data amount of the current cursor interval in a unit interval based on the total amount of data and a unit interval determined in advance, wherein the unit interval is a data range unit for measuring the data to be traversed; determining a number of unit intervals corresponding to a next cursor interval according to a pre-designed calculation scheme based on the average data amount corresponding to the unit interval and a traversal magnitude threshold set in advance, wherein the pre-designed calculation scheme comprises at least one of the following: performing proportional calculation on the average data amount and the traversal magnitude threshold; performing integral operation on the average data amount with respect to the unit interval and multiplying the traversal magnitude threshold; performing differential operation on the average data amount with respect to the unit interval and multiplying the traversal magnitude threshold; determining the next cursor interval based on the number of unit intervals corresponding to the current cursor interval and the next cursor interval.

2. The method of claim 1, wherein, The calculation formula of performing proportional calculation on the average data amount and the traversal magnitude threshold is: wherein i identifies the current cursor interval, i+1 identifies the next cursor interval, an average data amount corresponding to the interval unit corresponding to the current cursor interval, a number of interval units corresponding to the next cursor interval, a proportional control coefficient, a pre-set traversal magnitude threshold.

3. The method of claim 1, wherein, The calculation formula of performing integral operation on the average data amount with respect to the unit interval and multiplying the traversal magnitude threshold is: wherein i identifies the current cursor interval, i+1 identifies the next cursor interval, an average data amount corresponding to the interval unit corresponding to the current cursor interval, a number of interval units corresponding to the next cursor interval, an integral control coefficient, a pre-set traversal magnitude threshold, x indicating an interval unit.

4. The method of claim 1, wherein, The calculation formula of performing differential operation on the average data amount with respect to the unit interval and multiplying the traversal magnitude threshold is: wherein i identifies the current cursor interval, i+1 identifies the next cursor interval, an average data amount corresponding to the interval unit corresponding to the current cursor interval, a number of interval units corresponding to the next cursor interval, a differential control coefficient, a pre-set traversal magnitude threshold, x indicating an interval unit.

5. The method of claim 1, wherein, The determining of the current cursor interval corresponding to the data to be traversed, and the determining of the total amount of data traversed based on traversing the data in the current cursor interval by the cursor, comprises: performing paging operation on the current cursor interval according to a preset page size to obtain a plurality of data pages; traversing the plurality of data pages by an incremental cursor one by one until the size of the data page traversed is less than the preset page size to end the traversal; determining the total amount of data traversed based on the traversal result.

6. A data screening method, comprising: obtaining a first screening condition and a second screening condition, wherein the first screening condition is a condition for screening by a database index, and the second screening condition is a condition that cannot be screened by the database index; performing query operation corresponding to the first screening condition based on the database index to obtain data to be traversed; obtaining a current cursor interval corresponding to the data to be traversed, and traversing the data in the current cursor interval by a cursor based on the second screening condition as a query condition to obtain data matching the second screening condition, and determining a total amount of data traversed; calculating an average data amount of the current cursor interval in a unit interval based on the total amount of data and a unit interval determined in advance, wherein the unit interval is a data range unit for measuring the data to be traversed; According to the average data amount corresponding to the interval unit and a preset traversal magnitude threshold, a quantity of interval units corresponding to a next cursor interval is determined according to a preset calculation scheme, wherein the preset calculation scheme comprises at least one of the following: performing proportional calculation on the average data amount and the traversal magnitude threshold; performing integral operation on the average data amount with respect to the interval unit and multiplying the traversal magnitude threshold; performing differential operation on the average data amount with respect to the interval unit and multiplying the traversal magnitude threshold; determining the next cursor interval according to the quantities of interval units corresponding to the current cursor interval and the next cursor interval, and continuing to perform the steps of taking the second filtering condition as a query condition, traversing data in the current cursor interval based on a cursor, obtaining data matching the second filtering condition, and determining the total amount of data that is traversed.

7. A database system, comprising: a data interval traversal module configured to determine a current cursor interval corresponding to data to be traversed, and determine a total amount of data that is traversed by traversing data in the current cursor interval based on a cursor; a data interval stable division module configured to calculate an average data amount of the current cursor interval in an interval unit according to the total amount of data and a preset interval unit, wherein the interval unit is a data range unit for measuring the data to be traversed, and a quantity of interval units corresponding to a next cursor interval is determined according to a preset calculation scheme based on the average data amount corresponding to the interval unit and a preset traversal magnitude threshold, wherein the preset calculation scheme comprises at least one of the following: performing proportional calculation on the average data amount and the traversal magnitude threshold; performing integral operation on the average data amount with respect to the interval unit and multiplying the traversal magnitude threshold; performing differential operation on the average data amount with respect to the interval unit and multiplying the traversal magnitude threshold; and determining the next cursor interval according to the quantities of interval units corresponding to the current cursor interval and the next cursor interval.

8. An electronic device comprising: a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface being in communication with each other through the communication bus; the memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the method of any one of claims 1-6.

9. A computer storage medium having a computer program stored thereon, the program being executed by a processor to implement the method of any one of claims 1-6.

Images