Business data processing method and electronic device

Abstract

The application discloses a service data processing method and electronic equipment, and relates to the technical field of computers, and the method comprises the following steps: in a current service data processing round, a plurality of service data corresponding to a timing task is acquired from a service database; according to priorities corresponding to the plurality of service data, the service data is divided to obtain at least one service data set; according to the priorities corresponding to the service data in a first service data set, the number of threads in a server allocated for the first service data set is determined, wherein the first service data set is any one of the at least one service data set; and according to the number of threads allocated for the first service data set, the service data in the first service data set is processed. Through the application, the number of threads is dynamically determined according to the priorities of the service data, the server thread resources are reasonably utilized, and the service data processing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a business data processing method and electronic device. Background Technology

[0002] In information-based business systems, scheduled tasks are crucial for automating business data processing. By pre-setting execution cycles or trigger conditions and based on pre-defined processing logic, they process business data to automate business processes and ensure continuous and stable operations. However, as business scale continues to expand, the amount of data that scheduled tasks need to process also increases dramatically, leading to low processing efficiency and making it difficult to meet the needs of current business scenarios. Summary of the Invention

[0003] This application provides a business data processing method and electronic device to at least solve the problem of low efficiency in processing data by scheduled tasks.

[0004] This application provides a business data processing method, including:

[0005] In the current business data processing round, retrieve multiple business data corresponding to the scheduled task from the business database;

[0006] Based on the priority of each business data item, the business data items are divided to obtain at least one business data set.

[0007] The number of threads in the server allocated to the first business data set is determined based on the priority of the business data in the first business data set, wherein the first business data set is any one of at least one business data set;

[0008] The business data in the first business data set is processed according to the number of threads allocated to the first business data set.

[0009] This application also provides a business data processing apparatus, including:

[0010] The acquisition module is used to retrieve multiple business data corresponding to the scheduled task from the business database in the current business data processing round.

[0011] The partitioning module is used to partition the business data according to the priority of each business data, so as to obtain at least one business data set.

[0012] The determining module is used to determine the number of threads in the server allocated to the first business data set based on the priority of the business data in the first business data set, wherein the first business data set is any one of at least one business data set;

[0013] The processing module is used to process the business data in the first business data set according to the number of threads allocated to the first business data set.

[0014] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described business data processing methods.

[0015] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described business data processing methods.

[0016] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described business data processing methods.

[0017] This application utilizes the priority of acquired business data to divide multiple business data sets, dynamically determining the number of threads allocated to each set based on its priority. For example, more threads are allocated to high-priority business data. This avoids resource imbalances on the server, such as using a large number of threads to process low-priority business data or insufficient threads preventing timely processing of high-priority business data, thus achieving rational allocation of server resources and reducing waste. Furthermore, matching the corresponding number of threads to business data of different priorities for parallel processing fully leverages the concurrency advantages of multithreading, accelerating the overall processing speed and improving efficiency. In addition, business data sets containing different priorities are processed through independent thread pools, achieving isolation between different priorities and preventing abnormal processing of business data from affecting the processing of other priorities. For instance, when a processing anomaly occurs in a certain priority data (especially low-priority business data) (such as logical errors or execution blocking), its impact is limited to its own thread pool and does not affect the processing of other priority data, effectively ensuring the stability and continuity of high-priority core business data processing. Attached Figure Description

[0018] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1A flowchart illustrating a business data processing method provided in this application embodiment;

[0020] Figure 2 A flowchart illustrating yet another business data processing method provided in this application embodiment;

[0021] Figure 3 This is a schematic diagram of the structure of a business data processing device provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0024] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0025] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] First, the application scenarios of the embodiments of this application will be introduced by way of example.

[0027] In a distributed cluster environment, cluster monitoring and scheduled tasks are two core functions ensuring the stable operation of the distributed environment. Cluster monitoring is responsible for tracking the physical status (such as node liveness and hardware load) and performance status (such as CPU utilization, memory usage, and network throughput) of the entire cluster in real time. By promptly detecting anomalies such as node crashes, service failures, and resource exhaustion, it effectively avoids cascading failures. Simultaneously, based on long-term accumulated monitoring data, it can accurately analyze system bottlenecks, optimize resource allocation strategies, and improve overall stability. Scheduled tasks, on the other hand, undertake periodic business processing responsibilities, such as periodically executing log cleanup, database index optimization, model cache refresh, and data processing. Their core objective is to avoid resource contention during peak hours by staggering execution times, reducing the risk of accidental deletion or missed executions caused by human error, and ultimately achieving full utilization of cluster resources, completing the supplementation and correction of business data without affecting user experience.

[0028] Taking an AI platform as an example, it provides core services such as inference computation, model training, and algorithm deployment. In such complex distributed cluster environments, AI platforms rely on Kubernetes (k8s) for cluster resource management and scheduling, and use Docker containerization technology to deploy scheduled tasks. Scheduled tasks can be used to process business data, back up file systems, and so on. For example, during platform operation, business data (such as disk space usage and CPU usage time) is continuously stored in the business database. Scheduled tasks then extract and process this business data according to a preset cycle, generating corresponding invoices and other information.

[0029] However, when dealing with large volumes of business data, scheduled tasks may fail to complete processing within the preset time, resulting in task failure. Related technologies address this by extending the execution duration of the scheduled task or re-starting it during off-peak hours (e.g., early morning). However, extending the execution duration can disrupt subsequent tasks, and re-starting the task during off-peak hours can delay the processing of corresponding business data, preventing the timely acquisition of processing results such as invoices. This impacts the timeliness of business data processing, the efficiency of platform resource utilization, and fails to meet the demands for high response speed and stability.

[0030] In view of this, embodiments of this application provide a business data processing method to improve the processing efficiency of business data and rationally allocate thread resources.

[0031] It should be noted that the business data processing method provided in the embodiments of the present invention can be executed by a business data processing device. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The electronic device can be a server or a terminal. In the embodiments of this application, the server can be a single server or a server cluster composed of multiple servers. The terminal in the embodiments of this application can be a smartphone, personal computer, tablet computer, wearable device, or other intelligent hardware device such as an intelligent robot. In the following method embodiments, the execution subject is always described using an electronic device as an example.

[0032] According to an embodiment of the present invention, a business data processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This embodiment provides a business data processing method that can be used in the aforementioned electronic devices, such as servers. Figure 1 This is a flowchart of a business data processing method provided according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes:

[0034] S101: In the current business data processing round, retrieve multiple business data corresponding to the scheduled task from the business database.

[0035] Specifically, scheduled tasks refer to tasks that are automatically executed according to preset times or conditions (such as log cleanup, data backup, data statistics, etc.). For example, scheduled tasks can be divided into several levels based on their execution frequency, such as Level 1, Level 2, Level 3, and Level 4. Level 1 has shorter intervals, typically executing every 5 or 15 minutes. Level 2 executes hourly, such as every 1 or 2 hours. Level 3 executes daily, usually around 2 AM (this is typically the time with the least business activity and more idle cluster resources). Level 4 executes weekly.

[0036] The current business processing round refers to the business data processing process that is executed once according to a preset cycle or conditions after a scheduled task is triggered. For example, the bill calculation based on business data performed every day at 10:00 AM.

[0037] Business data refers to the various types of business data that scheduled tasks need to process. For example, taking a scheduled task as a computational invoice, the business data in the business database can be related to information such as the computing resources used based on the task request. Taking a model training request as an example, the business data in the business database can include the time of the task request, the duration of model training, and the CPU, bandwidth, and memory resources used during training. Business data can be categorized according to the type of task request it corresponds to. For example, business data can be categorized as business data corresponding to model training requests, inference requests, algorithm requests, and script editing requests.

[0038] In one possible implementation, after the scheduled task is triggered, the server connects to the business database and executes an SQL query to obtain multiple business data corresponding to the scheduled task.

[0039] S102, based on the priority of each business data, divide the business data into at least one business data set.

[0040] The priority of business data is determined according to its importance or urgency. For example, business data corresponding to inference requests has a higher priority than business data corresponding to model training requests.

[0041] For example, priority labels are pre-assigned to each piece of business data. Using data classification algorithms, high-priority, medium-priority, and low-priority business data are grouped into different business data sets. This allows for differentiated processing based on the varying urgency and importance of different business data sets. It prevents the processing of high-priority business data from being blocked by low-priority data, ensuring orderly processing of business data and improving the processing speed of core business data.

[0042] S103, determine the number of threads in the server allocated to the first business data set based on the priority of the business data in the first business data set.

[0043] The first business data set can be any one of at least one business data set.

[0044] In one possible implementation, the number of threads allocated to the first business data set in the server can be determined based on a preset mapping relationship between priority and thread count, and the priority of the business data in the first business data set. For example, if the business data in the first business data set has a high priority, four threads are allocated to the first business data set; if the business data in the first business data set has a medium priority, three threads are allocated to the first business data set; and if the business data in the first business data set has a low priority, one thread is allocated to the first business data set.

[0045] This is because, considering that high-priority business data requires more resources to ensure processing efficiency and avoid delays due to insufficient threads, low-priority business data can reduce thread usage and avoid wasting server resources, thereby achieving precise allocation of thread resources and improving server resource utilization.

[0046] In another possible implementation, besides basing the allocation of threads on the priority of business data in the first business data set, the number of threads can also be dynamically adjusted based on the quantity of business data in the first business data set. For example, if the amount of data in the first business data set exceeds a preset data volume threshold, the number of threads can be increased.

[0047] S104, Process the business data in the first business data set according to the number of threads allocated to the first business data set.

[0048] Specifically, based on the number of threads allocated to the first business data set, threads in the server are allocated to the first business data set, and the business data in the first business data set is processed by the threads in the server allocated to the first business data set.

[0049] In this embodiment, multiple business data sets are obtained by prioritizing the acquired business data. The number of threads allocated to each business data set is dynamically determined based on its priority. For example, more threads are allocated to high-priority business data. This avoids uneven resource allocation on the server, such as using a large number of threads to process low-priority business data or insufficient threads causing high-priority business data to be unprocessed in a timely manner. This achieves reasonable allocation of server resources, reduces resource waste, and shortens the processing time for high-priority business data from minutes to seconds. Furthermore, matching a corresponding number of threads to business data of different priorities for parallel processing fully leverages the concurrency advantages of multithreading, accelerating the overall processing speed of business data and improving processing efficiency. In addition, business data sets containing business data of different priorities are processed through independent thread pools, achieving isolation between business data of different priorities and preventing the processing of abnormal business data from affecting the processing of business data of other priorities. For example, when a processing exception occurs in a certain priority data (especially low priority business data) (such as logical error or execution blockage), its impact is limited to its own thread pool and will not affect the processing of other priority data, effectively ensuring the stability and continuity of high priority core business data processing.

[0050] In some embodiments, based on the foregoing embodiments, before processing the business data in the first business data set according to the number of threads allocated to the first business data set, the method provided in this application embodiment further includes the following:

[0051] First, construct a pre-defined data table.

[0052] Specifically, the preset data table is used to store data related to scheduled tasks, such as business data corresponding to scheduled tasks, and processing results obtained by the server after processing the business data using threads.

[0053] Then, the business data corresponding to the scheduled tasks are stored in a preset data table.

[0054] Finally, retrieve the business data from the first business data set from the preset data table.

[0055] Understandably, the preset data table is a temporary table, and the server will delete it after the scheduled task has finished executing. For example, the preset data table might store business data corresponding to the scheduled task, and be deleted when the scheduled task's execution duration is reached. Alternatively, the preset data table might store not only business data but also the processing results of that business data. When the scheduled task's execution duration is reached, the processing results are stored from the preset data table into a preset database (such as a billing database), and then the preset data table is deleted.

[0056] By storing multiple business data used to execute scheduled tasks in a pre-defined data table, the number of frequent reads from different tables in the business data can be reduced, thereby reducing the access pressure on the business database.

[0057] In one possible implementation, the first business data set includes business data of at least one pre-defined task that has been completed and business data of at least one pre-defined task that has not been completed.

[0058] Specifically, business data for pre-defined tasks that have been completed refers to the business data generated after the pre-defined tasks (such as model training tasks and inference tasks) have been completed, including the task execution time and all computing resources used during the task execution, such as CPU and memory. Business data for pre-defined tasks that have not been completed refers to the business data generated by pre-defined tasks that are currently being executed (such as during model training), including the task execution time and the computing resources used by the task.

[0059] For example, in the business database, business data corresponds to an execution status identifier for a specific task, indicating the completion status of that task. For instance, a high-priority business data set might contain both business data corresponding to a completed model A (e.g., all computing resources used during the training process) and business data corresponding to a model B currently being trained (already used computing resources). This way, in the current round of business data processing, determining the execution status of the tasks corresponding to the business data within the set provides a basis for processing multiple business data sets. For example, business data for pre-defined tasks that have already been executed can be processed first to ensure that core result data is processed preferentially.

[0060] In one possible scenario, based on the foregoing embodiments, in S104 above, the business data in the first business data set is processed according to the number of threads allocated to the first business data set, specifically including the following steps:

[0061] a1, based on the thread allocated to the first business data set, prioritizes processing the business data of each preset task that has been completed.

[0062] Optionally, in a1 above, business data of at least one pre-defined task that has been completed is processed in the following manner, including:

[0063] First, based on the number of business data for preset tasks that have been completed in the first business data set and the number of threads allocated to the first business data set, determine the number of business data for preset tasks that have been completed for each thread.

[0064] For example, the ratio between the number of business data in the first business data set that have completed the execution of preset tasks and the number of threads allocated to the first business data set is used as the number of business data for each thread that have completed the execution of preset tasks.

[0065] For example, if the first set of business data contains 50 completed business data items, and 10 threads are allocated to this set, then by division (50 ÷ 10 = 5), each thread needs to process 5 business data items. If there are any remaining business data items, they are randomly distributed to the aforementioned threads. This ensures that the workload of each thread is relatively balanced, avoiding some threads being idle or overloaded due to uneven load, improving thread resource utilization, and shortening the overall processing time.

[0066] Then, based on the amount of business data of the preset tasks that have been completed and processed by the first thread, the corresponding business data of the preset tasks that have been completed and processed are allocated to the first thread.

[0067] The first thread is any one of the threads assigned to the first business data set.

[0068] For example, a queue mechanism can be used to allocate a corresponding amount of business data to each thread. For instance, 10 task queues can be created for 10 threads, and business data from 50 pre-defined tasks that have been completed can be sequentially placed into the task queues according to a round-robin allocation principle (i.e., the first piece of business data is allocated to thread 1, the second piece to thread 2, and so on). This explicit allocation mechanism avoids thread contention for data, such as multiple threads simultaneously reading the same business data, thus binding threads to data, reducing data conflicts, and ensuring the reliability of the data processing process.

[0069] Finally, the business data of the preset tasks that have been completed and executed in the first thread are processed through the first thread.

[0070] For example, the bill corresponding to the business data is determined by the first thread based on the business data of the preset tasks that have been executed and the preset design fee rules corresponding to the business data.

[0071] For example, the billing rule for "model training tasks" could be "hourly GPU usage cost × training duration + data storage cost". Similarly, the billing rule for "inference tasks" could be "cost per inference request × total number of inference requests".

[0072] a2, after all the business data of the pre-set tasks that have been completed have been processed, then process the business data of each pre-set task that has not been completed.

[0073] For example, four threads are allocated to high-priority business data. These four threads first process all business data within the high-priority business data that has completed pre-defined tasks, such as calculating invoices using the business data from completed pre-defined tasks. After all the business data from completed pre-defined tasks has been processed, the same four threads are then used to process the business data from the high-priority business data that has not yet completed pre-defined tasks.

[0074] In this way, business data that has completed pre-defined tasks are processed first, avoiding competition for resources with business data that has not completed pre-defined tasks, improving resource utilization, and ensuring reliable processing of business data that has completed pre-defined tasks.

[0075] Optionally, in a2 above, the business data of each pre-defined task that has not been completed is processed in the following manner:

[0076] First, the business data of each pre-defined task that has not been completed is marked, resulting in marked business data.

[0077] Then, the tagged business data is stored in the business database.

[0078] Of course, when the scheduled task's execution time is reached, unprocessed business data can be marked and stored in the business database. This way, when the next scheduled task starts—the next business data processing round—the server will prioritize processing the marked unprocessed business data, as well as the business data of the marked but unfinished preset tasks.

[0079] Figure 2 This is a flowchart illustrating yet another business data processing method. Figure 2In this process, once a scheduled task begins, firstly, multiple business data corresponding to the scheduled task are read from the business database and stored in a preset data table. Secondly, based on the priority of each business data item, the multiple business data items are divided into multiple business data sets, and the number of threads allocated to each business data set is determined according to the priority of the business data within each set. Then, based on the number of threads allocated to each business data set, business data items in the set whose preset tasks have been completed are processed first, followed by business data items in the set whose preset tasks have not been completed. Next, when the execution duration corresponding to the scheduled task is reached, it is determined whether all business data items in the business data set have been processed. If there is unprocessed business data in the business data set, the processing results corresponding to the processed business data are read from the preset data table and stored in a preset database (such as a billing database). At the same time, the unprocessed business data is marked and stored in the business database. If there is no unprocessed business data in the business data set, the processing results corresponding to the processed business data are read from the preset data table and stored in the preset database (such as a billing database). Finally, the preset data table and the allocated threads are reclaimed.

[0080] Optionally, the method provided in this application embodiment also includes the following:

[0081] In the next round of business data processing, if the acquired business data is tagged, the tagged business data will be processed first.

[0082] For example, in the next business data processing round, the server first retrieves multiple business data from the business database. If there is marked business data among these data (such as marked but unprocessed business data or marked business data with incomplete pre-defined tasks), the marked business data is processed first. After the marked business data is processed, the business data in the first business data set is processed according to the number of threads allocated to the first business data set. In this way, marked business data is processed first in each business data processing round, avoiding the backlog of newly added data in subsequent rounds and preventing it from remaining unprocessed for a long time, thus improving the integrity and reliability of business data processing.

[0083] Optionally, in a1 above, after processing the business data of each completed preset task according to the thread allocated to the first business data set, the method provided in this application embodiment further includes the following:

[0084] After the first business data in the first business data set is processed, the processing result corresponding to the first business data is stored in a preset data table.

[0085] The first business data is any one of the business data of at least one pre-defined task that has been completed.

[0086] Taking a scheduled task as an example of calculating a bill, the processing result corresponding to the first business data can be a bill based on the first business data.

[0087] Optionally, the method provided in this application embodiment also includes the following:

[0088] First, when the execution time corresponding to the scheduled task is reached, the processing result corresponding to the processed business data is read from the preset data table.

[0089] Then, the processing results corresponding to the read and processed business data are stored in the preset database.

[0090] In related technologies, each time a piece of business data is read from a business database, it is processed, and the processing result is written to a preset database, such as a billing database. Then, the process continues, reading the next piece of business data from the business database. Each write operation to the preset database causes a table lock, which is only released after the write operation is complete. In the current round of business data processing, multiple pieces of business data need to be processed to obtain the processing result for each piece of business data. Therefore, multiple reads from the business database and multiple writes to the preset database occur, leading to frequent table locks in the preset database and increasing the frequency of memory garbage collection. To avoid frequent table locks and reduce the locking behavior of the preset database, thus reducing the frequency of memory garbage collection, in this embodiment, after the first piece of business data is processed, the processing result corresponding to the first piece of business data is stored in a preset data table. Only when the execution duration corresponding to the scheduled task is reached is the processing result corresponding to all processed business data retrieved from the preset data table and stored in the preset database. In this way, the preset data table is used as a temporary storage carrier. The processing results corresponding to a single business data will not be directly written to the preset database, that is, the table locking mechanism of the preset database will not be triggered. By writing all processing results to the preset database in batches, the original method of writing to the database once for processing one piece of data is simplified to writing once after batch processing, thereby reducing the frequency of table locking and memory garbage collection.

[0091] In some embodiments, the method provided in this application further includes the following steps:

[0092] c1, during the process of each thread processing the business data in the first business data set, obtains the processing speed corresponding to each thread.

[0093] Specifically, thread processing speed refers to the number of business data items processed per unit of time while processing business data in the first set of business data. For example, threads 1 and 2 handle high-priority business data; thread 1 can process 5 business data items per second, and thread 2 can process 3 business data items per second.

[0094] For example, within a preset time period, the amount of business data processed by the thread is obtained, and the ratio between this amount and the preset time period is used as the processing speed of the thread.

[0095] c2 determines the thread processing speed corresponding to the first business data set based on the processing speed of each thread.

[0096] Specifically, the thread processing speed corresponding to the first business data set is the overall speed obtained by summing the processing speeds of all threads processing the first business data set, reflecting the processing efficiency of each thread on the first business data set. For example, if there are 3 threads processing the first business data set with processing speeds of 5 records / second, 3 records / second, and 4 records / second respectively, the thread processing speed corresponding to the first business data set can be the average of the processing speeds of the three threads, i.e., 4 records / second.

[0097] c3. When the thread processing speed corresponding to the first business data set is not within the preset speed range corresponding to the first business data set, the number of threads allocated to the first business data set is adjusted according to the thread processing speed corresponding to the first business data set and the preset speed range.

[0098] Specifically, the preset speed range can be determined based on the data characteristics in the first business data set, such as data complexity and priority. This application embodiment does not specifically limit this. For example, the preset speed range for high-priority business data sets is [5 data points / second, 10 data points / second], and the preset speed range for medium-priority business data sets is [3 data points / second, 6 data points / second].

[0099] In one possible implementation, the number of threads allocated to the first business data set is adjusted based on the thread processing speed corresponding to the first business data set and a preset speed range. Specific steps include:

[0100] First, based on the thread processing speed and the preset speed range, determine the speed difference corresponding to the first business data set.

[0101] Specifically, the speed difference refers to the difference between the thread processing speed corresponding to the first business data set and the boundary value of the preset speed range, which is used to quantify the degree to which the processing speed deviates from the reasonable range.

[0102] Optionally, when the thread processing speed is less than the lower limit of the preset speed range, the speed difference is the difference between the lower limit of the preset speed range and the thread processing speed. When the thread processing speed is greater than the upper limit of the preset speed range, the speed difference is the difference between the thread processing speed and the upper limit of the preset speed range.

[0103] For example, if the preset speed range is 3-6 pieces / second and the thread processing speed is 2.5 pieces / second, the speed difference is the difference between the lower limit of the preset speed range and the thread processing speed, i.e., 3-2.5=0.5 pieces / second; if the thread processing speed is 7 pieces / second, the speed difference is the difference between the thread processing speed and the upper limit of the preset speed range, 7-6=1 pieces / second.

[0104] Then, based on the speed difference and the preset speed range, the number of threads allocated to the first business data set is adjusted.

[0105] Optionally, when the thread processing speed is less than the lower limit of the preset speed range, the number of threads to be added to the first business data set is determined based on the speed difference, the lower limit of the preset speed range, and the number of threads currently allocated to the first business data set.

[0106] For example, the number of threads added to the first business data set is determined by the following formula:

[0107]

[0108] in, To increase the number of threads, The number of threads currently allocated for the first business data set. The speed difference This is the lower limit of the preset speed range. For example, when the thread processing speed is 2.5 threads / second and the preset speed range is [3 threads / second, 6 threads / second], the speed difference is 0.5 threads / second, the lower limit of the preset speed range is 3 threads / second, and the current number of threads allocated to the first business data set is 5, the number of threads added is 1, that is, the adjusted number of threads allocated to the first business data set is 6.

[0109] Optionally, when the thread processing speed is greater than the upper limit of the preset speed range, the number of threads to be reduced for the first business data set is determined based on the speed difference, the upper limit of the preset speed range, and the number of threads currently allocated to the first business data set.

[0110] For example, the number of threads reduced for the first business data set is determined by the following formula:

[0111]

[0112] in, To reduce the number of threads, The number of threads currently allocated for the first business data set. The speed difference This is the upper limit of the preset speed range. For example, when the thread processing speed is 7 threads / second and the preset speed range is [3 threads / second, 6 threads / second], the speed difference is 1 thread / second, the upper limit of the preset speed range is 6 threads / second, and the number of threads currently allocated to the first business data set is 8, the number of threads reduced is 1, that is, the number of threads allocated to the first business data set after adjustment is 7.

[0113] In this embodiment, the speed difference reflects the degree of mismatch between thread resources and processing requirements. Combined with the preset speed range, the number of threads allocated to the first business data set can be finely adjusted, enhancing the adaptability between threads and business data. That is, when the complexity of business data increases, causing the processing speed to decrease, threads are increased through speed difference adjustment; when data simplification leads to excessively high speed, threads are reduced, ensuring that the allocation of threads always adapts to the dynamic changes in business. This ensures that the processing speed meets the preset speed range as soon as possible (e.g., increasing the speed by adding threads when it is below the lower limit) while avoiding resource waste (e.g., reducing threads to release resources when it is above the upper limit), thus improving the accuracy and efficiency of the adjustment.

[0114] In some embodiments, the method provided in this application further includes the following:

[0115] When all business data in the first business data set has been processed, and there is unprocessed business data among various business data sets, the unprocessed business data is processed using the thread allocated to the first business data set.

[0116] For example, the unprocessed business data is sorted according to the priority of each unprocessed business data, and the unprocessed business data is processed using the thread allocated to the first business data set according to the sorting order of the unprocessed business data.

[0117] In this embodiment of the application, the thread allocated to the first business data set is in an idle state after completing its own processing task, while the unprocessed part of other business data still requires thread resources. If the thread is released directly, it will cause a waste of resources. Therefore, reusing idle threads can improve the processing efficiency of business data, shorten the processing time of scheduled tasks, and improve resource utilization.

[0118] For example, a scheduled task is executed daily at 10:00 AM to retrieve business data from 9:45 AM to 10:00 AM within a specific timeframe. Assume that in a certain data processing round, 13,250 data entries are retrieved, of which 3,800 are high-priority, 8,200 are medium-priority, and 1,250 are low-priority. The server allocates 4 threads to process high-priority data, 3 threads to process medium-priority data, and 1 thread to process low-priority data.

[0119] Taking a thread that processes high-priority business data as an example, it prioritizes processing business data for preset tasks that have been completed and stores the corresponding processing results in a preset data table, and then processes business data for preset tasks that have not been completed.

[0120] Assuming all threads process each type of business data at the same speed, each thread handling high-priority business data processes 950 data points, each thread handling low-priority business data processes 1250 data points, and each thread handling medium-priority business data processes 2734 data points. Therefore, after the high-priority business data is processed, the threads allocated for it continue processing the medium-priority business data. At this point, there are 7 threads processing medium-priority business data, with each thread processing an average of approximately 765 data points. The threads processing low-priority business data have only 300 data points remaining. It can be predicted that the low-priority data will be completed faster, and once that is finished, the medium-priority business data will be processed. At this point, 3250 medium-priority business data points remain unprocessed, and all 8 threads process this portion, with each thread processing an average of approximately 407 data points.

[0121] The above mainly describes the solution provided by the embodiments of this application from a methodological perspective.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0123] This application also provides a business data processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0124] This embodiment provides a business data processing device, such as... Figure 3 As shown, it includes:

[0125] The acquisition module 301 is used to retrieve multiple business data corresponding to the scheduled task from the business database in the current business data processing round.

[0126] The partitioning module 302 is used to partition the business data according to the priority of each business data to obtain at least one business data set;

[0127] The determining module 303 is used to determine the number of threads in the server allocated to the first business data set based on the priority of the business data in the first business data set, wherein the first business data set is any one of at least one business data set;

[0128] Processing module 304 is used to process business data in the first business data set according to the number of threads allocated to the first business data set.

[0129] In one possible implementation, the device further includes: a construction module and a storage module;

[0130] The building module is used to build the preset data table;

[0131] The storage module is used to store multiple business data corresponding to scheduled tasks into a preset data table;

[0132] The acquisition module 301 is also used to acquire business data from the first business data set from a preset data table.

[0133] In one possible implementation, the first business data set includes business data of at least one pre-defined task that has been completed and business data of at least one pre-defined task that has not been completed.

[0134] The processing module 304 is specifically used to prioritize processing the business data of each preset task that has been completed, based on the thread allocated for the first business data set.

[0135] After all the business data of the pre-defined tasks that have been completed have been processed, the business data of each pre-defined task that has not been completed will be processed.

[0136] In one possible implementation, the storage module is further configured to store the processing result corresponding to the first business data in the first business data set into a preset data table after the first business data in the first business data set has been processed, wherein the first business data is any one of the business data of at least one preset task that has been completed.

[0137] In one possible implementation, the processing module 304 is specifically used to mark the business data of each unfinished preset task to obtain the marked business data;

[0138] The tagged business data is stored in the business database.

[0139] In one possible implementation, the acquisition module 301 is further configured to read the processing result corresponding to the processed business data from a preset data table when the execution duration corresponding to the scheduled task is reached; the storage module is further configured to store the processing result corresponding to the read processed business data into a preset database.

[0140] In one possible implementation, the processing module 304 is also used to prioritize processing the marked business data in the next business data processing round when the obtained business data is marked business data.

[0141] In one possible implementation, the acquisition module 301 is also used to acquire the processing speed of each thread during the process of each thread processing the business data in the first business data set.

[0142] The determining module 303 is also used to determine the thread processing speed corresponding to the first business data set based on the processing speed of each thread; when the thread processing speed corresponding to the first business data set is not within the preset speed range corresponding to the first business data set, the number of threads allocated to the first business data set is adjusted according to the thread processing speed corresponding to the first business data set and the preset speed range.

[0143] In one possible implementation, the determining module 303 is specifically used to determine the speed difference corresponding to the first business data set based on the thread processing speed and a preset speed range.

[0144] The number of threads allocated to the first business data set is adjusted based on the speed difference and the preset speed range.

[0145] The apparatus provided in this application divides multiple business data sets by prioritizing the acquired business data. It then dynamically determines the number of threads allocated to each set based on its priority. For example, more threads are allocated to high-priority business data. This avoids resource imbalances on the server, such as using a large number of threads to process low-priority business data or insufficient threads preventing timely processing of high-priority business data, thus achieving a reasonable allocation of server resources and reducing waste. Furthermore, matching a corresponding number of threads to business data of different priorities for parallel processing fully leverages the concurrency advantages of multithreading, accelerating the overall processing speed and improving efficiency. In addition, business data sets containing different priorities are processed through independent thread pools, achieving isolation between different priorities and preventing abnormal processing of business data from affecting the processing of other priorities. For instance, when a certain priority data (especially low-priority business data) experiences a processing anomaly (such as a logical error or execution blockage), its impact is limited to its own thread pool and does not affect the processing of other priority data, effectively ensuring the stability and continuity of high-priority core business data processing.

[0146] For a description of the features in the embodiment corresponding to the business data processing device, please refer to the relevant description in the embodiment corresponding to the business data processing method, which will not be repeated here.

[0147] Embodiments of this application also provide an electronic device, such as... Figure 4 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described business data processing method embodiments.

[0148] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described business data processing method embodiments at runtime.

[0149] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0150] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described business data processing method embodiments.

[0151] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described business data processing method embodiments.

[0152] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0153] The above provides a detailed description of a business data processing method and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A business data processing method, characterized in that, Applied to a server, the method includes: In the current business data processing round, retrieve multiple business data corresponding to the scheduled task from the business database; Based on the priority corresponding to each of the multiple business data, the business data is divided to obtain at least one business data set; Based on the priority corresponding to the business data in the first business data set, the number of threads in the server allocated to the first business data set is determined, wherein the first business data set is any one of at least one of the business data sets; The business data in the first business data set is processed according to the number of threads allocated to the first business data set; The method further includes: During the process of each thread processing the business data in the first business data set, the processing speed corresponding to each thread is obtained. Based on the processing speed of each thread, determine the thread processing speed corresponding to the first business data set; When the processing speed of the thread corresponding to the first business data set is not within the preset speed range corresponding to the first business data set, the number of threads allocated to the first business data set is adjusted according to the processing speed of the thread corresponding to the first business data set and the preset speed range. When the thread processing speed corresponding to the first business data set is not within the preset speed range corresponding to the first business data set, adjusting the number of threads allocated to the first business data set based on the thread processing speed corresponding to the first business data set and the preset speed range includes: Based on the thread processing speed and the preset speed range, determine the speed difference corresponding to the first business data set; When the thread processing speed is less than the lower limit of the preset speed range, the number of threads to be added to the first business data set is determined based on the speed difference, the lower limit of the preset speed range, and the number of threads currently allocated to the first business data set. or, When the thread processing speed exceeds the upper limit of the preset speed range, the number of threads to be reduced for the first business data set is determined based on the speed difference, the upper limit of the preset speed range, and the number of threads currently allocated to the first business data set. The number of threads added to the first business data set is determined using the following formula: in, To increase the number of threads, The number of threads currently allocated for the first business data set. The speed difference This is the lower limit of the preset speed range; The first business data set includes business data of at least one pre-defined task that has been completed and business data of at least one pre-defined task that has not been completed; the step of processing the business data in the first business data set according to the number of threads allocated to the first business data set includes: Based on the thread allocated to the first set of business data, the business data of each of the preset tasks that have been completed are processed first. After all the business data of the pre-defined tasks that have been completed have been processed, the business data of each pre-defined task that has not been completed is then processed.

2. The method according to claim 1, characterized in that, Before processing the business data in the first business data set according to the number of threads allocated to the first business data set, the method further includes: Build a pre-defined data table; Multiple business data corresponding to the scheduled task are stored in the preset data table; Retrieve business data from the first business data set from the preset data table.

3. The method according to claim 2, characterized in that, After processing the business data of each completed preset task according to the thread allocated to the first business data set, the method further includes: After the first business data in the first business data set is processed, the processing result corresponding to the first business data is stored in the preset data table, wherein the first business data is any one of the business data of at least one preset task that has been completed.

4. The method according to claim 1 or 3, characterized in that, Processing the business data for each of the aforementioned unfinished preset tasks includes: The business data of each of the aforementioned unfinished preset tasks is marked to obtain the marked business data; The tagged business data is stored in the business database.

5. The method according to claim 3, characterized in that, The method further includes: When the execution duration corresponding to the scheduled task is reached, the processing result corresponding to the processed business data is read from the preset data table; The processing results corresponding to the read and processed business data are stored in a preset database.

6. The method according to claim 4, characterized in that, The method further includes: In the next round of business data processing, if the acquired business data is the marked business data, the marked business data will be processed first.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the business data processing method as described in any one of claims 1-6 when executing the computer program.

Images