Database data row statistics method and device, equipment and storage medium

Abstract

Embodiments of the present application disclose a database data row number counting method and device, equipment and a storage medium, comprising: when counting a row number of target data in a target database of an electronic device, acquiring an index range corresponding to the target data; wherein the target data is data meeting a counting condition in all to-be-counted data contained in the target database; generating a plurality of counting tasks according to the index range; wherein each counting task is used for counting the row number of the target data in a partial index range; distributing the plurality of counting tasks to at least one thread created in advance for processing to obtain a plurality of counting results; and adding the plurality of counting results to obtain a target counting result of the row number of the target data. Through implementation of the method, the counting speed of the row number of data meeting the counting condition can be improved for a large database with a data volume of ten million or more when the index cannot be compressed.

Description

Technical Field

[0001] This application relates to the field of database technology, and to, but is not limited to, a method, apparatus, device, and storage medium for counting the number of rows in a database. Background Technology

[0002] Database row count is a common requirement in internet applications. It refers to counting the number of data points in a database that meet specific criteria, and can be used for data analysis, trend prediction, and report generation. When performing database row count, the query scope can be narrowed by creating indexes on specific fields, thus improving the speed of the calculation.

[0003] However, for databases containing large amounts of data, especially tens of millions or more, if the scope of the index is not small enough, the database will abandon index queries and perform full table scan queries when querying and statistically analyzing data, resulting in abnormally slow query and statistical analysis speeds.

[0004] Therefore, improving the efficiency of counting rows of specified data in databases with tens of millions or more of data is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the database data row count counting method, apparatus, device, and storage medium provided in this application embodiment can improve the speed of counting data under specified conditions in large databases with tens of millions or more of data by using divide-and-conquer and multi-threading when the range of indexed fields cannot be compressed. The database data row count counting method, apparatus, device, and storage medium provided in this application embodiment are implemented as follows:

[0006] The database row count method provided in this application includes:

[0007] When counting the number of rows of target data in the target database of an electronic device, the index range corresponding to the target data is obtained; wherein, the target data is the data that meets the statistical conditions among all the data to be counted in the target database;

[0008] Based on the index range, multiple statistical tasks are generated; each statistical task is used to count the number of rows of target data within a portion of the index range.

[0009] The multiple statistical tasks are assigned to at least one pre-created thread for processing to obtain the multiple statistical results;

[0010] The multiple statistical results are summed to obtain the target statistical result for the number of rows of the target data.

[0011] In some embodiments, generating multiple statistical tasks based on the index range includes:

[0012] The index range is divided into multiple sub-index ranges;

[0013] Based on each of the subdivided index ranges, a corresponding statistical task is generated, resulting in the plurality of statistical tasks.

[0014] In some embodiments, the number of the at least one thread is multiple, and the process of assigning the multiple statistical tasks to the at least one pre-created thread for processing to obtain the multiple statistical results includes:

[0015] Based on the number of threads, the multiple statistical tasks are grouped for processing, resulting in multiple statistical task groups;

[0016] The multiple statistical tasks are grouped and assigned to multiple threads for processing to obtain the multiple statistical results.

[0017] In some embodiments, the step of grouping and assigning the plurality of statistical tasks to the plurality of threads for processing to obtain the plurality of statistical results includes:

[0018] Each thread creates a double-ended queue and groups and stores the multiple statistical tasks in the double-ended queue corresponding to each thread;

[0019] Each thread sequentially executes the statistical tasks within the statistical task group corresponding to the double-ended queue, and obtains the statistical results corresponding to the thread.

[0020] The multiple statistical results are obtained based on multiple statistical results corresponding to threads.

[0021] In some embodiments, each thread sequentially executes the statistical tasks within the statistical task group corresponding to the double-ended queue to obtain statistical results corresponding to the thread, including:

[0022] If all statistical tasks in the double-ended queue corresponding to the thread have been completed, retrieve the unexecuted statistical tasks from the tail of the double-ended queue corresponding to the target thread that has not yet been completed, and process the unexecuted statistical tasks.

[0023] The statistical results corresponding to the thread are obtained based on the statistical results of all statistical tasks in the double-ended queue corresponding to the thread and the statistical results of the statistical tasks that have not yet started execution.

[0024] In some embodiments, the index range is divided into multiple sub-index ranges, including:

[0025] According to preset weights, the index range is divided into multiple sub-index ranges, and the difference in the number of statistical data to be collected in the different sub-index ranges is less than a threshold.

[0026] In some embodiments, the step of retrieving unexecuted statistical tasks from the tail of the deque corresponding to the target thread if all statistical tasks in the deque corresponding to the thread have been completed includes:

[0027] When all statistical tasks in the double-ended queue corresponding to the thread have been completed, determine whether the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to a preset threshold.

[0028] When the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to the preset threshold, the unexecuted statistical tasks are retrieved from the tail of the double-ended queue corresponding to the target thread.

[0029] The database row counting device provided in this application embodiment includes:

[0030] The index acquisition module is used to acquire the index range corresponding to the target data when counting the number of rows of target data in the target database of the electronic device; wherein, the target data is the data that meets the statistical conditions among all the data to be counted in the target database;

[0031] The task generation module is used to generate multiple statistical tasks based on the index range; wherein each statistical task is used to count the number of rows of data within a portion of the index range;

[0032] The task execution module is used to assign the multiple statistical tasks to at least one pre-created thread for processing to obtain the multiple statistical results; and to add the multiple statistical results to obtain the target statistical result of the number of rows of the target data.

[0033] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0034] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method described in this application embodiment.

[0035] The database row count counting method, apparatus, computer device, and computer-readable storage medium provided in this application, when counting the number of rows of target data in a target dataset using an electronic device, obtain the index range corresponding to the target data. The target data is a set of data in the target database that meets the statistical conditions, and the index range is a field range used to accelerate the counting of target data. Based on the index range, multiple statistical tasks are generated to count the number of rows of target data within a portion of the index range, avoiding a single, large overall index method that would lead to slow counting speed. The generated multiple statistical tasks are assigned to at least one pre-created thread for execution, resulting in multiple statistical results. Since each result only contains the number of rows of target data within a portion of the index range, the multiple statistical results are compared to obtain the target statistical result for the number of rows of the target data.

[0036] In this way, based on the overall index range of the target data, multiple statistical tasks are generated and executed by threads, thus solving the technical problems raised in the background. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0038] Figure 1 An illustration of an application scenario for the database row count method provided in this application embodiment;

[0039] Figure 2 A flowchart illustrating an implementation of a database row count method provided in this application embodiment;

[0040] Figure 3 This is a schematic diagram illustrating the division of the index range into subdivided index ranges in the database row count method provided in this application embodiment;

[0041] Figure 4 A flowchart illustrating another implementation of the database row count method provided in this application embodiment;

[0042] Figure 5 A schematic diagram illustrating the inter-thread acquisition of statistical tasks in a database row count method provided in this application embodiment;

[0043] Figure 6 A flowchart illustrating an implementation of a database row count method provided in this application for obtaining statistical results by thread processing of statistical tasks;

[0044] Figure 7 A schematic diagram of the structure of the database row counting device provided in the embodiments of this application;

[0045] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0048] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0050] Counting the number of rows in a database based on specified criteria is a common requirement. This row counting enables functions such as data analysis, report generation, trend prediction, monitoring and early warning systems, and data quality monitoring.

[0051] To improve the efficiency of counting rows of specified data and to save computing resources and reduce computation time, index-based query methods are commonly used. Indexes are typically created for fields that can be divided into ranges, such as time, quantity, and employee ID, to address slow query performance. By indexing these fields, the database system can quickly locate data that meets specific range conditions using the index's sorting information. When performing row counting, the system only needs to locate the range that meets the conditions in the index, without needing to scan the entire dataset. This optimization can significantly shorten query time and improve statistical efficiency.

[0052] It's important to note that index creation needs to be tailored to specific query requirements and data patterns. Furthermore, index creation and maintenance increase storage space and can impact the performance of operations such as insert, update, and delete. When designing indexes, factors such as query frequency, query scope, data update frequency, and storage space must be considered to balance query performance and data maintenance needs.

[0053] However, when dealing with tens of millions or more of data, if the range of fields used for indexing in the query conditions is not small enough, meaning the query conditions cannot effectively filter the data using the index, the database query optimizer will determine that a full table scan is less costly than using an index query, and thus perform a full table scan. This will consume a lot of computing resources and time, resulting in an abnormally slow query. This situation is particularly prominent when performing row count statistics.

[0054] In view of this, embodiments of this application provide a method for counting the number of data rows in a database, which is applied to an electronic device. This electronic device may include, but is not limited to, tablet computers, laptops, PCs (Personal Computers), large servers, etc. The function implemented by this method can be achieved by a processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium.

[0055] Please see Figure 1 This illustration shows an application scenario of the database row count method provided in this application embodiment. Figure 1 As shown, the data in the initial table is filtered according to specified conditions to obtain the row count. It is understandable that... Figure 1 The filter table in the code is used to represent the filtered data content. In practical applications, when only the row count is needed, the filter table does not need to be generated.

[0056] In some embodiments, the target database is a database built by an open-source relational database management system that supports cross-platform operation and is widely used in applications of various sizes and types. It provides powerful features, including data storage, data retrieval, transaction management, security control, and high availability. Through the rich set of statements and methods provided by the database system, users can easily manipulate data and obtain the desired results.

[0057] When counting rows of data that meet specified conditions in a database, the simplest and most direct method is to perform a full table scan, which involves traversing the entire table to count the rows. This method does not rely on any indexes or other structures and can accurately obtain the total number of rows of the desired target data in the table. Full table scans are often used for small databases or tables, or in situations where time and computational requirements are not critical. However, when dealing with databases containing large amounts of data, using a full table scan can consume significant computational resources and time.

[0058] At this point, an index query can be used. An index is a data structure used to improve query performance, helping database systems quickly locate and access specific rows in a table without traversing the entire table. For example... Figure 1 The initial table can have an index created on the "Time" field to determine the time range and speed up row counting. In the database, indexes are represented by "INDEX" and can be created, deleted, and operated on using corresponding statements.

[0059] In some embodiments, the data table or database to be statistically analyzed contains tens of millions or more of data. If the method of directly creating an index for a specific field is used, and the data volume is large, when the scope of the index is not clear enough, a large amount of data that meets the index conditions will appear. The index query still needs to scan a large number of data blocks, which will slow down the query and statistical query speed.

[0060] In summary, the embodiments of this application adopt the divide-and-conquer approach and provide a method for counting the number of data rows in a database. The range of fields used for indexing is divided into multiple smaller ranges after processing, and the number of rows of data in the database that meet the specified conditions is quickly counted through index queries.

[0061] Please see Figure 2 This is a flowchart illustrating one implementation of a database row count method provided in an embodiment of this application. Figure 2 As shown, the method may include steps 201 to 204.

[0062] In this embodiment of the application, the method for counting the number of data rows in the database includes:

[0063] When counting the number of rows of target data in the target database of an electronic device, the index range corresponding to the target data is obtained; wherein, the target data is the data that meets the statistical conditions among all the data to be counted in the target database;

[0064] Based on the index range, multiple statistical tasks are generated; each statistical task is used to count the number of rows of target data within a portion of the index range.

[0065] The multiple statistical tasks are assigned to at least one pre-created thread for processing to obtain the multiple statistical results;

[0066] The multiple statistical results are summed to obtain the target statistical result for the number of rows of the target data.

[0067] The methods provided in the embodiments of this application will be described step by step below.

[0068] Step 201: Obtain the index range corresponding to the target data.

[0069] In this embodiment of the application, when counting the number of rows of target data in the target database of the electronic device, the index range corresponding to the target data is obtained; wherein, the target data is the data that meets the statistical conditions among all the data to be counted in the target database.

[0070] It should be noted that the statistical conditions are pre-set data filtering restrictions used to select data that meets the actual application requirements. These conditions do not change during a single database row count. The index range corresponding to the target data can be entered by the user, obtained by marking the data in the database during input and using those marks, or obtained through analysis of the database data using appropriate algorithms; no specific limitations are imposed here. When using algorithms to obtain the index range, due to the large volume of data in the database, it is advisable to analyze a portion of the data and estimate the index range corresponding to the target data.

[0071] Step 202: Generate multiple statistical tasks based on the index range.

[0072] In this embodiment of the application, multiple statistical tasks are generated according to the index range; wherein each statistical task is used to count the number of rows of target data within a certain index range.

[0073] It should be noted that after obtaining the index range, the electronic device generates a corresponding database query statement based on the index range to count the number of rows of the target data. In order to facilitate the execution of subsequent threads and connection to the target database, the database query statement needs to be encapsulated to obtain the statistical task.

[0074] Step 203: Assign multiple statistical tasks to at least one pre-created thread for processing to obtain multiple statistical results.

[0075] In this embodiment of the application, the plurality of statistical tasks are assigned to at least one pre-created thread for processing to obtain the plurality of statistical results.

[0076] It should be noted that a thread is the smallest unit of task scheduling in an operating system, executing multiple statistical tasks in an electronic device. The number of at least one thread created in advance should be less than the number of multiple statistical tasks generated in the above steps. That is, when allocating multiple statistical tasks, each thread is assigned at least one statistical task to prevent some threads from not being assigned any statistical tasks, thus avoiding a waste of computing resources.

[0077] Step 204: Add the multiple statistical results together to obtain the target statistical result for the number of rows of the target data.

[0078] In this embodiment of the application, the multiple statistical results are added together to obtain the target statistical result of the number of rows of the target data.

[0079] It should be noted that since each statistical task is used to count the number of rows of target data within a certain index range, in order to obtain the total number of rows of all target data in the target database, it is necessary to add up the statistical results obtained from multiple statistical tasks to obtain the total target statistical result of the number of rows of target data.

[0080] In the above scheme, the index range corresponding to the target data is obtained, multiple statistical tasks are generated, and the multiple statistical tasks are assigned to at least one pre-created thread to execute and obtain multiple statistical results. The multiple statistical results are added together to obtain the target statistical result. This can improve the speed of counting rows of data that meet the statistical conditions for large databases with tens of millions or more of data volume when the index range cannot be compressed.

[0081] The process of dividing a large index range into multiple sub-index ranges in the method provided in the embodiments of this application will be described below with reference to the accompanying drawings and examples.

[0082] Please see Figure 3 This is a schematic diagram illustrating the method for counting the number of data rows in a database provided in this application, in which the index range is divided into subdivided index ranges.

[0083] like Figure 3 As shown, a global time [T1, T] t The time frame is divided into multiple sub-time ranges, and database query statements are generated based on these sub-time ranges and encapsulated into tasks for submission to threads for processing.

[0084] It should be noted that, Figure 3 The section describes the division of the overall time range. In some embodiments, the index range can also be other numerical values ​​that can be further subdivided, such as employee number, serial number, quantity, etc. Furthermore, character-type data such as model number and name can be converted into numerical form using a conversion algorithm, and then the range can be further subdivided. The index is selected according to actual needs and is not limited here.

[0085] In some embodiments, a time field is selected for index querying to count the number of target data that meet the conditions, i.e., the number of rows of target data in the database. Since for large databases with tens of millions or more of data, a large index range can make the statistical query extremely slow, the method in this application embodiment divides the large index range.

[0086] exist Figure 3 In some embodiments, the index field selected is "time", and the overall time range selected for indicating row count is [T1, T2]. t Directly querying this time range using an index would be slow. Therefore, this overall time range is divided into n sub-time ranges, including: [T1, T2), [T2, T3), ..., [T...]. t-2 ,T t-1 ), [T t-1 ,T t There are a total of n time ranges. Based on the n sub-time ranges and the filtering criteria for the target data, n SQL statements are generated, including: SQL1, SQL2, ..., SQL2. n-1 SQL n It's important to note that SQL is a computer language used to manage relational database systems. It's widely used for operations such as querying, inserting, updating, and deleting data. SQL has a simple and easy-to-understand syntax, helping users retrieve data from the database, modify data, and perform various database management tasks. After obtaining n SQL statements, they are encapsulated into statistical tasks and assigned to threads for execution. The number of SQL statements n is greater than or equal to the number of threads m, ensuring that each thread is assigned at least one statistical task, avoiding wasted computational resources. The multiple statistical results are then summed to obtain the final target statistical result.

[0087] In some embodiments, when using the "time" field as an index, the overall time range can be divided into several sufficiently small equal parts, meaning that each sub-time range represents the same duration. This partitioning method is simple to operate and suitable for databases where data is evenly distributed across different time periods. However, when the data distribution is uneven across different time periods, it can cause significant differences in the amount of data across different sub-time ranges. This can lead to some threads completing their tasks while others are left with a large amount of unprocessed data, wasting computational resources. Even if a thread can acquire tasks from other threads, it is prone to competition for statistical tasks, further wasting computational resources.

[0088] In some application embodiments, the index range is divided into multiple sub-index ranges, including: dividing the index range into multiple sub-index ranges according to a preset weight, wherein the difference in the number of statistical data to be collected in the different sub-index ranges is less than a threshold.

[0089] It should be noted that the difference in the number of statistical data items included in multiple sub-index ranges is less than the threshold. This means that the difference in the amount of data included in any two sub-index ranges after the division cannot exceed the threshold. The purpose is to ensure that the data can be evenly distributed in the statistical tasks generated after the sub-index ranges are divided.

[0090] In some embodiments, when the "time" field is used as the index, the overall time range can be divided into sub-time ranges of different sizes by pre-setting weights to ensure that the amount of data contained in each sub-time range is balanced and that the difference in the amount of data between multiple sub-time ranges is not too large.

[0091] Taking agricultural product sales data as an example, if a fruit's natural ripening time in the sales area is from June to August, and other times it can only be obtained through transportation from other regions or greenhouse cultivation, it's understandable that, under normal circumstances, the sales data for this fruit will be concentrated within the three months of natural ripening. In this case, it's necessary to statistically analyze the sales performance of this type of fruit over recent years to guide subsequent planting. When dividing the overall time range, using the above-mentioned equal-length time interval division method will result in the data volume corresponding to the sub-time range containing sales data from June to August being much larger than the data volume corresponding to other sub-time ranges. This can lead to some threads completing their tasks while others have a large amount of uncollected data to process, wasting computational resources. Alternatively, the data volume in a single sub-time range may be too large, resulting in slow index queries. Users can preset weights based on experience to divide the overall time range. Due to the massive amount of data in the database, a partial data set can also be used to calculate the weight distribution of data within a certain range using deep learning algorithms, thereby estimating the overall data and obtaining the final preset weights. The method for obtaining the preset weights is not limited here.

[0092] It should be noted that, in Figure 3 In the above embodiments, the "time" field is used as an index for illustrative purposes. In practical applications, other fields that can be divided into ranges can be selected as indexes and the above method can be implemented according to application requirements. No limitation is made here.

[0093] The above content explains the process of dividing the index range into multiple sub-index ranges in the embodiments of this application. The following, with reference to the accompanying drawings, will further explain the optimization of computational resource utilization and statistical speed in the database row count method provided in the embodiments of this application, using a work-stealing algorithm between threads.

[0094] Please see Figure 4 This is a flowchart illustrating another implementation of the database row count method provided in this application embodiment. Figure 4 As shown, the method may include steps 401 to 404.

[0095] In some embodiments, the multiple statistical tasks are grouped according to the number of threads to obtain multiple statistical task groups; the multiple statistical task groups are then assigned to the multiple threads for processing to obtain the multiple statistical results.

[0096] The methods provided in the embodiments of this application will be described step by step below.

[0097] Step 401: Obtain the index range corresponding to the target data.

[0098] Step 402: Generate multiple statistical tasks based on the index range.

[0099] Step 403: Based on the number of pre-created threads, the multiple statistical tasks are grouped for processing, resulting in multiple statistical task groups.

[0100] It should be noted that the number of statistical task groups obtained by grouping multiple statistical tasks should be greater than or equal to the number of threads to avoid wasting computing resources. Furthermore, the difference in the number of statistical tasks contained in different threads should be less than a preset threshold.

[0101] Step 404: Group the multiple statistical tasks and assign them to multiple threads for processing to obtain multiple statistical results.

[0102] In some embodiments, each thread creates a deque and stores the plurality of statistical tasks in a deque corresponding to each thread; each thread sequentially executes the statistical tasks in the statistical task group corresponding to the deque to obtain the statistical results corresponding to the thread; and the plurality of statistical results are obtained based on the plurality of statistical results corresponding to the threads.

[0103] It should be noted that the sequential execution of statistical tasks within the corresponding statistical task group of the double-ended queue by each thread means that when the statistical task group is stored in the double-ended queue, the statistical tasks within the group are sorted. When a thread executes the task corresponding to the double-ended queue, it starts executing the statistical task from the head of the double-ended queue, and the subsequent statistical tasks are executed sequentially according to the queue order.

[0104] Step 405: Add the multiple statistical results together to obtain the target statistical result for the number of rows of the target data.

[0105] In the above technical solution, multiple statistical tasks are grouped for processing, and the resulting multiple statistical tasks are assigned to multiple threads for simultaneous processing, thereby improving the speed of counting rows of the target data.

[0106] The following description, in conjunction with the accompanying drawings, illustrates the process by which a thread that has completed all corresponding statistical tasks retrieves the statistical tasks from threads that have not yet executed statistical tasks in the method provided in this application embodiment.

[0107] Please see Figure 5 This is a schematic diagram illustrating the inter-thread acquisition of statistical tasks in the database row count method provided in this application embodiment.

[0108] like Figure 5 As shown, in some embodiments, there are m threads and m deques corresponding to the threads, and each deque contains several statistical tasks. A thread can start executing the statistical tasks in its corresponding deque from the head of the deque. As can be seen in the figure, thread m-1 retrieves the unexecuted statistical task (2.3) from the tail of deque 2 corresponding to thread 2 and places it at the tail of deque m-1. That is, a thread can retrieve unexecuted statistical tasks from the tail of other threads' deques and place them in its own deque.

[0109] The following diagram illustrates the process by which a thread obtains statistical tasks from a target thread.

[0110] Please see Figure 6 This is a flowchart illustrating an implementation of a database row count method provided in this application, where a thread processes a statistical task to obtain statistical results. Figure 6 As shown, the method may include steps 601 to 607.

[0111] It is understandable that when performing statistical analysis on target data in databases with tens of millions or more of data, even when generating statistical tasks and distributing them to multiple threads, and even if the allocation steps use relevant algorithmic steps to maintain a balance of data volume for each thread, it is still possible that some threads will complete all tasks while others still have a large number of statistical tasks to be executed. This situation can be further amplified due to device limitations or statistical conditions. To prevent threads from entering an idle state and wasting computing resources after completing their corresponding statistical tasks, the following steps will be taken to enable threads to obtain tasks from target threads that still have statistical tasks, thereby improving the speed of counting rows of target data.

[0112] The methods provided in the embodiments of this application will be described step by step below.

[0113] Step 601: The thread establishes a double-ended queue and stores the statistical tasks in the double-ended queue.

[0114] Step 602: Execute the statistical tasks within the statistical task groups in the double-ended queue sequentially and record the statistical results of each statistical task.

[0115] Step 603: Determine whether all statistical tasks in the double-ended queue corresponding to the thread have been completed.

[0116] In some embodiments, if all statistical tasks in the double-ended queue corresponding to a thread have been completed, the statistical tasks that have not yet started execution are retrieved from the tail of the double-ended queue corresponding to the target thread that has not yet been completed, and the statistical tasks that have not yet started execution are processed.

[0117] By determining whether all statistical tasks in the corresponding double-ended queue of a thread have been completed, it decides whether to retrieve unstarted statistical data from the target thread's queue. It should be noted that selecting unstarted statistical tasks ensures the accuracy of the final target statistical results; selecting already started statistical tasks would result in multiple threads performing duplicate statistics.

[0118] If the condition is met, proceed to step 604. If the condition is not met, proceed to step 602 until all statistical tasks corresponding to the thread have been completed.

[0119] Step 604: Select the target thread for which there is an unexecuted statistical task.

[0120] Step 605: Determine whether the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to a preset threshold.

[0121] In some embodiments, when all statistical tasks in the double-ended queue corresponding to the thread have been completed, it is determined whether the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to a preset threshold.

[0122] It should be noted that selecting the target thread to acquire statistical tasks based on whether the number of unexecuted statistical tasks in the double-ended queue is greater than or equal to a preset threshold is to prevent other threads from acquiring statistical tasks when the target thread has only a few remaining tasks. This would cause the target thread to also need to acquire tasks from outside, leading to contention among threads and unnecessary waste of computing resources.

[0123] When a thread retrieves unexecuted statistical tasks from the tail of the double-ended queue corresponding to the target thread, it can retrieve one statistical task or multiple statistical tasks at once; there is no limitation on this. It must be ensured that the number of unexecuted statistical tasks within the target thread after retrieval is greater than or equal to a preset threshold.

[0124] If the judgment result of step 605 is "yes", then proceed to step 606; if it is "no", then proceed to step 603 to switch threads to obtain the target thread of the statistics task.

[0125] Step 606: Obtain the statistical tasks that have not yet started execution from the tail of the double-ended queue corresponding to the target thread, and execute them to obtain the statistical results.

[0126] Step 607: Based on the statistical tasks within the statistical task group corresponding to the thread and the statistical tasks obtained from the target thread, process the sum of the obtained statistical results to obtain the statistical result corresponding to the thread.

[0127] In the above technical solution, the thread that has completed all statistical tasks obtains the statistical tasks from the double-ended queue of the target threads that have not yet executed statistical tasks. This avoids the waste of computing resources caused by some threads not performing any other processing after completing the statistical tasks of the corresponding statistical task group.

[0128] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0129] Based on the foregoing embodiments, this application provides a database row count counting device. The device includes various modules and units included in each module, which can be implemented by a processor; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.

[0130] Please see Figure 7 This is a schematic diagram of the structure of the database row counting device provided in the embodiments of this application, as shown below. Figure 7As shown, the device 700 includes an index acquisition module 701, a task generation module 702, and a task execution module 703, wherein:

[0131] The index acquisition module 701 is used to acquire the index range corresponding to the target data when counting the number of rows of target data in the target database of the electronic device; wherein, the target data is the data that meets the statistical conditions among all the data to be counted in the target database.

[0132] The task generation module 702 is used to generate multiple statistical tasks based on the index range; wherein each statistical task is used to count the number of rows of data within a portion of the index range;

[0133] The task execution module 703 is used to assign the plurality of statistical tasks to at least one pre-created thread for processing to obtain the plurality of statistical results; and to add the plurality of statistical results to obtain the target statistical result of the number of rows of the target data.

[0134] In some embodiments, the task generation module 702 is further configured to divide the index range into multiple sub-index ranges; generate corresponding statistical tasks according to each sub-index range, thereby obtaining the multiple statistical tasks.

[0135] In some embodiments, the task generation module 702 is further configured to divide the index range into multiple sub-index ranges according to a preset weight, wherein the difference in the number of statistical data to be collected in the multiple sub-index ranges is less than a threshold.

[0136] In some embodiments, the task execution module 703 is further configured to: group the plurality of statistical tasks according to the number of the plurality of threads to obtain a plurality of statistical task groups; and assign the plurality of statistical task groups to the plurality of threads for processing to obtain the plurality of statistical results.

[0137] In some embodiments, the task execution module 703 is further configured to: each thread create a double-ended queue and store the plurality of statistical tasks in the double-ended queue corresponding to each thread; each thread sequentially executes the statistical tasks in the statistical task group corresponding to the double-ended queue to obtain the statistical results corresponding to the thread; and obtain the plurality of statistical results based on the plurality of statistical results corresponding to the thread.

[0138] In some embodiments, the task execution module 703 is further configured to: if all statistical tasks in the double-ended queue corresponding to the thread have been completed, obtain the statistical tasks that have not yet started execution from the tail of the double-ended queue corresponding to the target thread that has not yet been completed, process the statistical tasks that have not yet started execution; and obtain the statistical results corresponding to the thread based on the statistical results of all statistical tasks in the double-ended queue corresponding to the thread and the statistical results of the statistical tasks that have not yet started execution.

[0139] In some embodiments, the task execution module 703 is further configured to, when all statistical tasks in the double-ended queue corresponding to the thread have been completed, determine whether the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to a preset threshold; and when the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to the preset threshold, obtain statistical tasks that have not yet started execution from the tail of the double-ended queue corresponding to the target thread.

[0140] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0141] It should be noted that, in the embodiments of this application... Figure 7 The data row counting device for the database shown is illustrative of the module division, representing only one logical functional division. In actual implementation, other division methods are possible. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit with two or more units. The integrated units can be implemented in hardware, as software functional units, or a combination of both.

[0142] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0143] This application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the methods described above.

[0144] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0145] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0146] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one embodiment, the database row counting device provided in this application can be implemented as a computer program, which can be implemented in the form of, for example, Figure 8 The device operates on the computer device shown. The memory of the computer device can store the various program modules that make up the above-described apparatus. The computer program, composed of the various program modules, causes the processor to execute the steps of the methods in the various embodiments of this application described in this specification.

[0148] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0149] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0150] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0151] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0153] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0154] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0155] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0156] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0157] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0158] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0159] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0160] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for counting the number of rows in a database, characterized in that, The method includes: When counting the number of rows of target data in the target database of an electronic device, the index range corresponding to the target data is obtained; wherein, the target data is the data that meets the statistical conditions among all the data to be counted in the target database; Based on the index range, multiple statistical tasks are generated; each statistical task is used to count the number of rows of target data within a portion of the index range. The multiple statistical tasks are assigned to at least one pre-created thread for processing to obtain the multiple statistical results; The multiple statistical results are added together to obtain the target statistical result for the number of rows of the target data; When there are multiple threads, the process of assigning the multiple statistical tasks to at least one pre-created thread for processing to obtain the multiple statistical results includes: Based on the number of threads, the multiple statistical tasks are grouped for processing, resulting in multiple statistical task groups; Each thread creates a double-ended queue and groups and stores the multiple statistical tasks in the double-ended queue corresponding to each thread; Each thread sequentially executes the statistical tasks within the statistical task group corresponding to the double-ended queue, and obtains the statistical results corresponding to the thread. The multiple statistical results are obtained based on multiple statistical results corresponding to threads.

2. The method according to claim 1, characterized in that, The process of generating multiple statistical tasks based on the index range includes: The index range is divided into multiple sub-index ranges; Based on each of the subdivided index ranges, a corresponding statistical task is generated, resulting in the plurality of statistical tasks.

3. The method according to claim 1, characterized in that, Each thread sequentially executes the statistical tasks within the statistical task group corresponding to the double-ended queue, obtaining statistical results corresponding to the thread, including: If all statistical tasks in the double-ended queue corresponding to the thread have been completed, retrieve the unexecuted statistical tasks from the tail of the double-ended queue corresponding to the target thread that has not yet been completed, and process the unexecuted statistical tasks. The statistical results corresponding to the thread are obtained based on the statistical results of all statistical tasks in the double-ended queue corresponding to the thread and the statistical results of the statistical tasks that have not yet started execution.

4. The method according to claim 2, characterized in that, The index range is divided into multiple sub-index ranges, including: According to preset weights, the index range is divided into multiple sub-index ranges, and the difference in the number of statistical data to be collected in the different sub-index ranges is less than a threshold.

5. The method according to claim 3, characterized in that, If all statistical tasks in the double-ended queue corresponding to the thread have been completed, the unexecuted statistical tasks are retrieved from the tail of the double-ended queue corresponding to the target thread that has not yet been completed, including: When all statistical tasks in the double-ended queue corresponding to the thread have been completed, determine whether the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to a preset threshold. When the number of unexecuted statistical tasks in the double-ended queue of the target thread is greater than or equal to the preset threshold, the unexecuted statistical tasks are retrieved from the tail of the double-ended queue corresponding to the target thread.

6. A database row count counting device, characterized in that, include: The index acquisition module is used to acquire the index range corresponding to the target data when counting the number of rows of target data in the target database of the electronic device; wherein, the target data is the data that meets the statistical conditions among all the data to be counted in the target database; The task generation module is used to generate multiple statistical tasks based on the index range; wherein each statistical task is used to count the number of rows of data within a portion of the index range; The task execution module is used to assign the multiple statistical tasks to at least one pre-created thread for processing to obtain the multiple statistical results; and to add the multiple statistical results to obtain the target statistical result of the number of rows of the target data. The task execution module is further configured to, when the number of the at least one thread is multiple, group the multiple statistical tasks according to the number of multiple threads to obtain multiple statistical task groups; each thread creates a double-ended queue and stores the multiple statistical task groups in the double-ended queue corresponding to each thread; each thread sequentially executes the statistical tasks in the statistical task group corresponding to the double-ended queue to obtain the statistical results corresponding to the thread; and obtains the multiple statistical results based on the multiple statistical results corresponding to the threads.

7. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images