Scheduling method and system for power transaction data collection task

Abstract

The application relates to the technical field of power transaction data collection and task scheduling and provides a scheduling method and system for power transaction data collection tasks. The scheduling method comprises the following steps: based on multiple identifications of power transaction data collection tasks, corresponding queue type channels and corresponding concurrency strategies are selected from multiple queue type channels; and the scheduling and dispatching of the power transaction data collection tasks are performed by using the corresponding queue type channels and the corresponding concurrency strategies. The scheduling method of the application can support multiple accounts, strong wind control constraints and time-sensitive application scenarios in a power transaction market, and is beneficial to supporting large-scale concurrent scheduling and low fault tolerance requirements.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to the field of power trading data acquisition and task scheduling technology, and in particular to a scheduling method and system for power trading data acquisition tasks. Background Technology

[0002] In the electricity trading market, there are numerous accounts and stringent risk control constraints. Electricity trading data collection may require concurrent scheduling of hundreds of trading accounts and thousands of scheduled tasks, thus presenting many challenges unique to the electricity trading data collection industry. For example, provincial electricity trading centers are sensitive to a large number of concurrent requests or frequent requests within a short period from the same account after login, which may trigger risk control measures such as logging out, requiring verification codes, temporary bans, or disconnections. Furthermore, a single electricity trading account may initiate similar disclosure queries, such as price inquiries or offers, to multiple potential counterparties. These batches of homogeneous requests can cause queue backlogs and uneven load distribution, hindering overall throughput. Finally, multiple electricity trading accounts may need to collaborate, such as procuring electricity from multiple sources and then centrally supplying it to a specific region, thus posing unique requirements for the management of electricity trading accounts in the electricity trading market. Existing power dispatching solutions, such as Chinese patents with publication numbers CN122240315A, CN119135619A, and CN117479332A, lack consideration for the industry characteristics and specific needs of the power trading market and power trading data acquisition tasks, thus lacking effective technical means to address the aforementioned challenges. In the field of communication message and data forwarding, there are existing dispatching schemes based on distributed queues and dynamic balancing, such as Chinese patents with publication numbers CN115034792A and CN110225074B. However, these message and data forwarding dispatching schemes only provide a distributed task scheduling architecture for network data packets and service flows, without fully considering the strong risk control constraints and account collaboration characteristics of the power trading industry. Therefore, they are difficult to support the large-scale concurrent dispatching and low fault tolerance requirements of the power trading market.

[0003] To this end, this application proposes a scheduling method and system for power trading data acquisition tasks. By utilizing account queue isolation technology, account selection technology, rebalancing technology, and sharding and dispatching technology, it enables applications in the power trading market that support multiple accounts, strong risk control constraints, and time-sensitive applications, which is beneficial for supporting the needs of large-scale concurrent scheduling and low fault tolerance. Summary of the Invention

[0004] Firstly, this application provides a scheduling method for power trading data acquisition tasks. The scheduling method includes: selecting a corresponding queue type channel and a corresponding concurrency strategy from multiple queue type channels based on various identifiers of the power trading data acquisition task; and executing the scheduling and dispatch of the power trading data acquisition task using the corresponding queue type channel and the corresponding concurrency strategy. The multiple identifiers include at least a designated account identifier and a sharded account group identifier. The multiple queue type channels include at least multiple account-specific channels and sharded distribution channels that correspond one-to-one with multiple power trading accounts. The designated account identifier is used to select a designated power trading account among the multiple power trading accounts. The sharded account group identifier is used to select a list of distribution accounts. The list of distribution accounts is used to select at least two power trading accounts that are in the same sharded group among the multiple power trading accounts.

[0005] The first aspect of this application enables application scenarios in the power trading market that support multiple accounts, strong risk control constraints, and time sensitivity, which is conducive to supporting the needs of large-scale concurrent scheduling and low fault tolerance.

[0006] In one possible implementation of the first aspect of this application, the multiple account-specific channels all adopt a single-concurrency strategy, and the multiple account-specific channels are isolated from each other.

[0007] In one possible implementation of the first aspect of this application, the scheduling method further includes: when the designated account identifier indicates the designated power trading account, sending the power trading data acquisition task to the account-specific channel corresponding to the designated power trading account.

[0008] In one possible implementation of the first aspect of this application, the multiple queue type channels further include a default dispatch channel, and the scheduling method further includes: when the designated account identifier does not indicate any of the multiple power trading accounts, sending the power trading data collection task to the default dispatch channel; then, through the account selection module, selecting a candidate account-specific channel from the multiple account-specific channels based on a dual evaluation of queue depth scoring and historical time consumption estimation; and sending the power trading data collection task to the candidate account-specific channel.

[0009] In one possible implementation of the first aspect of this application, the scheduling method further includes: when the shard account group identifier indicates the distribution account list, limiting the power trading account corresponding to the candidate account exclusive channel to be in the distribution account list.

[0010] In one possible implementation of the first aspect of this application, the scheduling method further includes: determining, through a two-level task sharding module, whether the power trading data collection task is configured with the sharding account group identifier and whether the number of power trading accounts included in the dispatch account list after deduplication and login availability filtering is greater than one; if so, determining that the power trading data collection task has triggered the sharding trigger condition, and then sending the power trading data collection task as the parent task to the sharding dispatch channel.

[0011] In one possible implementation of the first aspect of this application, the scheduling method further includes: performing first-level account-level sharding through the two-level task sharding module to create multiple subtasks associated with the parent task, wherein each of the multiple subtasks is associated with the instance identifier of the parent task and the sharding sequence number corresponding to the dispatch account list, and each of the multiple subtasks is associated with the power trading accounts included in the same sharding group in the dispatch account list, respectively and without repetition.

[0012] In one possible implementation of the first aspect of this application, the scheduling method further includes: performing second-level request item-level sharding through the two-level task sharding module to deduplicate and remap the separable request items of the multiple subtasks, and maintaining that the same request item is mapped to the same sharding number and that request item sets under different sharding numbers are mutually exclusive.

[0013] In one possible implementation of the first aspect of this application, the scheduling method further includes: sending the plurality of subtasks to the account-specific channels corresponding to their respective associated power trading accounts, and updating the sharding progress of the parent task by detecting the progress of each of the plurality of subtasks.

[0014] In one possible implementation of the first aspect of this application, the outputs of each of the plurality of subtasks are written to a local sharding buffer, and after the sharding progress of the parent task is completed, the outputs of each of the plurality of subtasks stored in the local sharding buffer are merged and uploaded.

[0015] In one possible implementation of the first aspect of this application, the scheduling method further includes: when the power trading account associated with any of the plurality of subtasks is unavailable, selecting an untried power trading account from the list of dispatch accounts of the same shard group of the parent task for task reassignment.

[0016] In one possible implementation of the first aspect of this application, the plurality of identifiers further includes a rebalancing identifier, and the scheduling method further includes: periodically scanning the tasks in the waiting queues of the plurality of account-specific channels through a dynamic rebalancing module to determine whether there are any tasks to be migrated that meet the migration triggering conditions, and when it is determined that there are tasks to be migrated, performing task migration and rebalancing operations based on the rebalancing identifier of the task to be migrated and the shard account group identifier.

[0017] In one possible implementation of the first aspect of this application, the task to be migrated is forwarded through the default dispatch channel, and the migration triggering conditions include the task waiting time of the task to be migrated exceeding the minimum waiting threshold and the migration benefit of migrating the task to be migrated from the current power trading account to the candidate power trading account being higher than the migration benefit threshold.

[0018] In one possible implementation of the first aspect of this application, the task migration and rebalancing operation is performed based on the rebalancing identifier of the task to be migrated and the shard account group identifier, including: when the rebalancing identifier allows migration and the shard account group identifier indicates the assignment account list, selecting an available power trading account from the assignment account list of the same shard group for task reassignment; when the rebalancing identifier allows migration and the shard account group identifier does not indicate the assignment account list, selecting an available power trading account from the plurality of power trading accounts for task reassignment.

[0019] In one possible implementation of the first aspect of this application, the scheduling method further includes: when the designated account identifier indicates the designated power trading account and the parallel processing capability of the shard group to which the designated power trading account belongs does not meet the disclosure window timeliness, the account selection module selects a shard group that meets the capacity constraint from the regional account pool associated with the designated power trading account and enqueues it.

[0020] In one possible implementation of the first aspect of this application, the scheduling method further includes: when the designated account identifier indicates the designated power trading account and the designated power trading account forwards more than a threshold number of homogeneous requests associated with the power trading data collection task to the account-specific channel corresponding to the designated power trading account within a preset time period, merging the homogeneous requests as a parent task, and then sending the parent task to the sharding dispatch channel through a two-level task sharding module.

[0021] Secondly, embodiments of this application also provide an electronic device, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a method according to any of the above-mentioned implementations.

[0022] Thirdly, embodiments of this application also provide a computer-readable storage medium storing computer instructions that, when executed on a computer device, cause the computer device to perform a method according to any of the above-described implementations.

[0023] Fourthly, embodiments of this application also provide a computer program product, the computer program product including instructions stored on a computer-readable storage medium, which, when executed on a computer device, cause the computer device to perform a method according to any of the above-described aspects.

[0024] Fifthly, this application also provides a scheduling system for power trading data acquisition tasks. The scheduling system includes a multi-queue topology layer, an account selection layer, a dynamic rebalancing layer, a two-level task sharding layer, a high-availability reassignment layer, and an execution observation layer. The scheduling system is used to: through the account selection layer, select a corresponding queue type channel and a corresponding concurrency strategy from multiple queue type channels based on multiple identifiers of the power trading data acquisition task; and execute the scheduling and assignment of the power trading data acquisition task using the corresponding queue type channel and the corresponding concurrency strategy. The multiple identifiers include at least a designated account identifier and a sharded account group identifier; the multiple queue type channels include at least multiple account-specific channels and sharded assignment channels corresponding one-to-one with multiple power trading accounts; the designated account identifier is used to select a specific power trading account from the multiple power trading accounts; the sharded account group identifier is used to select an assignment account list; and the assignment account list is used to select at least two power trading accounts from the multiple power trading accounts that are in the same shard group. The multi-queue topology layer provides a default dispatch channel, multiple account-specific channels, and sharded dispatch channels. The account selection layer selects accounts based on queue status and historical latency. The dynamic rebalancing layer determines migration conditions and reassigns tasks. The two-level task sharding layer performs account-level and request-item-level sharding. The high-availability reassignment layer selects other accounts within the same shard group when an account fails. The execution observation layer records task execution time, migration count, failure summary, and results of missed or delayed executions.

[0025] The fifth aspect of this application enables application scenarios in the power trading market that support multiple accounts, strong risk control constraints, and time sensitivity, which is conducive to supporting the needs of large-scale concurrent scheduling and low fault tolerance. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating a scheduling method for power transaction data acquisition tasks provided in this application embodiment; Figure 2 A schematic diagram of a multi-queue topology isolated by account, provided for an embodiment of this application; Figure 3 A flowchart illustrating a method for account selection and load scoring provided in an embodiment of this application; Figure 4 A flowchart illustrating a two-level task sharding and output aggregation method provided in this application embodiment; Figure 5 A schematic diagram illustrating multiple paths for dynamic rebalancing and sibling account reassignment provided in an embodiment of this application; Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this application; Figure 7 This is a schematic diagram of a scheduling system for power transaction data acquisition tasks provided in an embodiment of this application. Detailed Implementation

[0028] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0029] It should be understood that in the description of this application, "at least one" means one or more, and "multiple" means two or more. In addition, the words "first," "second," etc., unless otherwise stated, are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0030] Figure 1 This is a flowchart illustrating a scheduling method for power transaction data acquisition tasks, provided as an embodiment of this application. Figure 1 As shown, the scheduling method includes the following steps.

[0031] Step S110: Based on the various identifiers of the power trading data acquisition task, select the corresponding queue type channel and the corresponding concurrency strategy from multiple queue type channels.

[0032] Step S120: Utilize the corresponding queue type channel and the corresponding concurrency strategy to schedule and dispatch the power trading data acquisition task.

[0033] See Figure 1The scheduling method shown includes at least a designated account identifier and a sharded account group identifier, and at least multiple queue type channels including multiple account-specific channels and sharded distribution channels corresponding one-to-one with multiple power trading accounts. The designated account identifier is used to select a specific power trading account from the multiple power trading accounts, the sharded account group identifier is used to select a list of distribution accounts, and the list of distribution accounts is used to select at least two power trading accounts from the multiple power trading accounts that are in the same shard group. In this way, the complex control problems faced in the power trading industry are transformed into a series of operations based on identifier-based data streams. Multiple identifiers are applied to the power trading data acquisition task, and appropriate queue type channels and corresponding concurrency strategies are selected based on these identifiers. Finally, task scheduling and distribution are executed. Here, considering the strong risk control constraints and account collaboration characteristics of the power trading industry, the multiple identifiers include at least a designated account identifier and a sharded account group identifier, and the multiple queue type channels include at least multiple account-specific channels and sharded distribution channels corresponding one-to-one with multiple power trading accounts. By using designated account identifiers, power trading data acquisition tasks can select a specific power trading account from among multiple power trading accounts, thus supporting single-account serial processing and improving security through a multi-queue topology isolated by account. Furthermore, using shard account group identifiers, power trading data acquisition tasks can be associated with a list of dispatching accounts and with power trading accounts within the same shard group. A shard group contains at least two power trading accounts, and these accounts can be considered as a single entity, used to handle bundled tasks. This fully utilizes the parallelism of the dedicated channels of each account within the same shard group, allowing for the packaging of multi-account collaborative tasks or highly similar tasks together. Then, the dispatching account list selects the dedicated channels corresponding to each of the multiple power trading accounts within the same shard group, and task scheduling and dispatch are performed through the shard dispatch channels. Therefore, by utilizing multiple identifiers, data collection tasks from various participants in the electricity trading market can be assigned appropriate queue types, channels, and corresponding concurrency strategies based on specific task types and requirements. For example, assignment can be done by specifying a power trading account and its corresponding dedicated channel, or by using a list of assigned accounts, at least two power trading accounts within the same shard group, and a sharded assignment channel. In this way, dedicated account channels support single-account serial processing to ensure risk control, while sharded assignment channels support cross-account parallel processing to guarantee overall throughput for batch homogeneous requests and multi-account collaboration.In addition, because it is based on the operation of the identifier data stream, it does not require a complex control system and control method. Through account queue isolation, account selection and sharding distribution, it supports application scenarios with multiple accounts, strong risk control constraints and time sensitivity in the power trading market, which is conducive to supporting the needs of large-scale concurrent scheduling and low fault tolerance.

[0034] See Figure 1 In one possible implementation of the scheduling method shown, the multiple account-specific channels all employ a single-concurrency strategy, and these channels are isolated from each other. This supports a multi-queue topology based on account isolation, improving system security. Furthermore, by adopting a single-concurrency strategy, it ensures that each power trading account has an independent execution channel and guarantees single-account serial execution from an architectural perspective, effectively mitigating concurrency risks on the target website.

[0035] See Figure 1 In one possible implementation of the scheduling method shown, the scheduling method further includes: when the designated account identifier indicates the designated power trading account, sending the power trading data acquisition task to the account-specific channel corresponding to the designated power trading account. This ensures single-account serialization at the architectural level, forwarding tasks to the corresponding account-specific channel to avoid duplicate consumption. Furthermore, utilizing a multi-queue topology isolated by account helps meet the stringent risk control constraints of the power trading industry.

[0036] In some embodiments, the multiple queue type channels also include a default dispatch channel, and the scheduling method further includes: when the specified account identifier does not indicate any of the multiple power trading accounts, sending the power trading data collection task to the default dispatch channel; then, through the account selection module, selecting a candidate account-specific channel from the multiple account-specific channels based on a dual evaluation of queue depth score and historical time consumption estimation; and sending the power trading data collection task to the candidate account-specific channel. Thus, when no power trading account is specified, the task can be forwarded through the default dispatch channel. Here, the default dispatch channel is only responsible for selecting the account and forwarding, and the subsequent process is completed through the selected candidate account-specific channel. Furthermore, after forwarding the task through the default dispatch channel, the task on the default dispatch channel side can be marked as completed to avoid duplicate consumption. Here, for power trading data collection tasks where the specified account identifier does not indicate any of the multiple power trading accounts, the account selection module uses a dual evaluation to select a candidate account channel. The dual evaluation includes queue depth score and historical time consumption estimation. In some examples, the input metrics for queue depth scoring include the number of waiting queue tasks for each candidate power trading account, the count of temporary placeholder tasks recently assigned but not yet reflected in the redistribution status within the current process, the number of tasks currently being executed, the number of delayed queue tasks, and whether currently executing tasks have been running continuously for more than a timeout threshold (e.g., 3 minutes). Thus, by assigning weights to the input metrics and performing a weighted sum, a unified scoring formula can be used to provide a queue depth score for each candidate power trading account. This queue depth score reflects the task execution and queue waiting status among each candidate power trading account. Furthermore, the queue depth score reflects the current status of a specific candidate power trading account. Based on the queue depth score, historical time consumption estimates are also incorporated; that is, by referencing historical data, the sum of the existing queue occupancy time for each candidate power trading account and the estimated time consumption of the currently pending tasks is further estimated as the total expected occupancy time. In some examples, historical time consumption estimates can refer to data including the remaining execution time of the current task, the sum of the estimated times of sampled tasks in the waiting queue, the time occupied by recently delayed tasks, the time occupied by tasks that have been assigned but not yet enqueued, and the estimated time of the current task to be dispatched. Here, the estimated time of the current task to be dispatched is preferentially read from the historical average execution time matched by the same task name, the same category, and the same account dimension. If historical data is lacking, a rollback operation can be performed according to the default value of the task category (e.g., the default execution time of daily report tasks is 120 seconds, and the execution time of monthly report tasks is 300 seconds) or the designed global minimum value (e.g., providing a minimum of 180 seconds to complete any task category).The account selection module uses a dual evaluation based on queue depth scoring and historical time consumption estimation to estimate the expected start time of each candidate power trading account for the current task. This allows for the convenient selection of the most suitable candidate power trading account and its corresponding dedicated account channel for subsequent processes.

[0037] In addition, queue-type channels include account-specific channels, default dispatch channels, and sharded dispatch channels. This supports serial processing within a single account to ensure risk control, while also supporting parallel processing across accounts to guarantee overall throughput for batch homogeneous requests and multi-account collaboration. In some examples, queue-type channels may also include real-time channels specifically for handling high-frequency real-time tasks, and isolated channels for executing special tasks individually. For details on queue-type channels and their corresponding concurrency strategies, please refer to Table 1 below: Table 1

[0038] See Figure 1 In one possible implementation of the scheduling method shown, the scheduling method further includes: when the shard account group identifier indicates the allocation account list, limiting the power trading account corresponding to the candidate account's dedicated channel to be within the allocation account list. Thus, if the task payload carries the allocation account list, the candidate account can be limited to the allocation account list, ensuring number selection and subsequent rebalancing. This also ensures that tasks carrying the same allocation account list are forwarded to the account-specific channel corresponding to the power trading account within the same shard group. This helps ensure that when tasks need to be reassigned later, they are also within the same shard group, which is beneficial for supporting priority rebalancing within the shard group and fully utilizing the parallelism of each account-specific channel while avoiding duplicate data collection across shard groups.

[0039] See Figure 1In one possible implementation of the scheduling method shown, the scheduling method further includes: determining, through a two-level task sharding module, whether the power trading data collection task is configured with the sharding account group identifier and whether the number of power trading accounts included in the dispatch account list after deduplication and login availability filtering is greater than one; if so, determining that the power trading data collection task has triggered the sharding trigger condition, and then sending the power trading data collection task as the parent task to the sharding dispatch channel. Thus, the two-level task sharding module determines whether the sharding trigger condition has been triggered at the entry point. When a task is configured with the sharding account group identifier and after deduplication and login availability filtering, if the number of accounts is greater than one, it is used as the parent task for the subsequent two-level sharding process. Here, when determining whether the sharding trigger condition has been triggered, tasks that have already been sharded as subtasks are also excluded through deduplication and login availability filtering. This allows for the provision of sharding dispatch channels for multi-account collaboration types or batch homogeneous requests, thereby improving overall throughput.

[0040] In some embodiments, the scheduling method further includes: performing first-level account-level sharding through the two-level task sharding module to create multiple subtasks associated with the parent task, wherein each subtask is associated with the instance identifier of the parent task and the shard number corresponding to the dispatch account list, and each subtask is independently and non-repeatingly associated with the power trading accounts included in the same shard group of the dispatch account list. Thus, after determining that the sharding trigger condition has been triggered, i.e., the number of power trading accounts included in the dispatch account list after deduplication and login availability filtering is greater than one, the first-level account-level sharding in the two-level sharding process is performed, creating multiple subtasks according to the dispatch account list. The identifier rule for each subtask can be the instance identifier of the parent task plus the account plus the shard number. Therefore, each subtask is associated with the instance identifier of the parent task and the shard number corresponding to the dispatch account list. Each subtask can carry more functional identifiers, such as whether the task is a subtask authentication identifier, the number of accounts within the same shard number and the same shard group.

[0041] In some embodiments, the scheduling method further includes: performing a second-level request item-level sharding through the two-level task sharding module to deduplicate and remap the separable request items of the multiple subtasks, while maintaining that the same request item is mapped to the same shard number and that request item sets under different shard numbers are mutually exclusive. Thus, based on the first-level account-level sharding, the second-level request item-level sharding allows filtering of the separable request item list when the subtask executes the collection logic. Specifically, this can be achieved by constructing a stable key for each request item, such as including the request item title and a sub-table from the original list, and performing a hash algorithm on the key, such as a string hash algorithm (e.g., character-by-character left shifting and absolute value taking). Using a hash modulo algorithm, the key can be modulo the total number of shards and equal to the current shard number, ensuring that the request item belongs to that shard for execution. Any suitable mapping algorithm can be used to deduplicate and remap the separable request items of the multiple subtasks, as long as the same request item is mapped to the same shard number and the request item sets under different shard numbers are mutually exclusive. This ensures that no duplicate data is collected during parallel processing across accounts. For example, suppose a contract type task includes 400 query items combining buyer and seller. Suppose four dispatch accounts are configured for this parent task. Then, each account's subtasks will execute approximately 100 items each after the mapping algorithm. The four accounts can process in parallel in four directions without overlap.

[0042] In some embodiments, the scheduling method further includes: sending the plurality of subtasks to the account-specific channels corresponding to their respective associated power trading accounts, and updating the sharding progress of the parent task by detecting the progress of each of the plurality of subtasks. Thus, after two-level sharding, requests that were originally completed individually by the parent task are distributed to various subtasks, which are then completed together by the subtasks of various dispatching accounts under the same parent task. Therefore, the subtasks of various dispatching accounts under the same parent task can be regarded as a sharding group, and the sharding progress of the parent task is updated by detecting the progress of each of the plurality of subtasks. For example, the parent task is responsible for maintaining a list of requests dispatched after two-level sharding, and using algorithms such as polling to wait for all subtasks to complete before updating the sharding progress percentage. The parent task enters a success state after all subtasks are successfully completed.

[0043] In some embodiments, the outputs of each of the multiple subtasks are written to a local shard buffer. After the shard progress of the parent task is completed, the outputs of each of the multiple subtasks stored in the local shard buffer are merged and uploaded. Thus, the output of each shard can first be written to a local directory named after the hash of the parent task identifier, and the metadata file records the total number of shards and the table header. When the number of shards written to disk equals the total number of shards, it means that all subtasks are completed. A distributed file lock is then acquired, and the data is concatenated in order according to the shard sequence number before uploading. Using the local disk as a shard buffer ensures that the number of accounts and the scale of requests that a single shard can handle are constrained by the instance's storage and processing capabilities, while also guaranteeing a predictable merging order. Thus, shard groups are first determined on the demand side according to the dispatching account (first level), and then mutually exclusive allocation is performed on the summarized splittable request items within each shard group (second level). Only after all shards are received are they merged and uploaded. This fully utilizes the parallelism of each account's dedicated channel while avoiding duplicate collection across shards.

[0044] In some embodiments, the scheduling method further includes: when the power trading account associated with any of the plurality of subtasks is unavailable, selecting an untried power trading account from the list of dispatch accounts in the same shard group of the parent task for task reassignment. Thus, subtasks of various dispatch accounts under the same parent task are considered a shard group, and various power trading accounts under the same shard number and the same shard group can be considered as sibling accounts. When a subtask under a shard group fails due to account unavailability (the reason for account unavailability may be authentication failure, account being kicked out, or being banned, etc.), a set of attempted accounts can be collected. Then, an untried sibling account is selected from the list of dispatch accounts in the same shard group of the parent task. By re-dispatching an equivalent subtask and updating the parent task synchronously, only the executing account is changed, but the request item hash mapping does not need to be modified, avoiding duplicate collection and mapping drift. Therefore, subtasks of various accounts under the same parent task constitute a shard group. When authentication fails for any account within the group, an untried sibling account is selected within the group instead of re-dividing the request item mapping. Furthermore, because task reassignment occurs within the same shard number and shard group, the impact of single account failure or prolonged task blocking is reduced. Therefore, the two-level task sharding module implements first-level account-level sharding and second-level request-item-level sharding, combining mutually exclusive allocation by account grouping and by request-item mapping. This allows hundreds to thousands of homogeneous query items to be parallelized across accounts in power trading scenarios and also supports multi-account collaboration scenarios. Additionally, when an account becomes unavailable and no available sibling accounts exist, the parent task can be marked as failed.

[0045] See Figure 1In one possible implementation of the scheduling method shown, the multiple identifiers further include a rebalancing identifier. The scheduling method also includes: periodically scanning the tasks in the waiting queues of the multiple account-specific channels using a dynamic rebalancing module to determine if there are any tasks to be migrated that meet the migration trigger conditions; and, when a task to be migrated is determined to exist, performing task migration and rebalancing operations based on the rebalancing identifier of the task to be migrated and the shard account group identifier. Thus, using the dynamic rebalancing identifier, the dynamic rebalancing module can periodically scan the tasks in the waiting queues of the multiple account-specific channels, for example, scanning once every 30 seconds. By detecting the existence of tasks to be migrated that meet the migration trigger conditions, task migration and rebalancing operations can be performed based on the rebalancing identifier of the task to be migrated and the shard account group identifier, thereby avoiding long-term task blocking and promoting load balancing. In some examples, the dynamic rebalancing module can also evaluate whether, when an account is marked as unavailable (e.g., authentication failure, account being kicked out, or banning), it is necessary to collectively migrate all public waiting tasks within that channel to the available account with the lowest usage, thus avoiding waiting for the shortest threshold. In addition, when an account is marked as unavailable, the migration logic and dynamic rebalancing module work together to make the unavailable mark time-limited and automatically removed when it expires.

[0046] In some embodiments, the task to be migrated is forwarded through the default dispatch channel. The migration triggering condition includes the task's waiting time exceeding a minimum waiting threshold and the migration benefit of migrating the task from the current power trading account to a candidate power trading account exceeding a migration benefit threshold. Thus, the migration triggering condition is set for tasks that meet both the waiting time and migration benefit requirements. By setting a minimum waiting threshold, such as 60 seconds, fluctuations immediately after task dispatch can be avoided. By setting a migration benefit threshold, such as requiring a minimum reduction of 30 seconds in waiting time, migration costs can be controlled and overall throughput improved. Additionally, the cumulative number of migrations for a single task can be limited to a certain number, such as no more than two times, to control the overall dynamic rebalancing burden.

[0047] In some embodiments, task migration and rebalancing operations are performed based on the rebalancing identifier and the shard account group identifier of the task to be migrated. This includes: when the rebalancing identifier allows migration and the shard account group identifier indicates the dispatch account list, selecting an available power trading account from the dispatch account list of the same shard group for task reassignment; and when the rebalancing identifier allows migration and the shard account group identifier does not indicate the dispatch account list, selecting an available power trading account from the plurality of power trading accounts for task reassignment. This ensures that the migration candidate range and the selection range are consistent. For tasks where the shard account group identifier indicates the dispatch account list, a better channel is prioritized within the original dispatch account list, i.e., within the range of sibling accounts in the same shard group. This helps coordinate with the two-level task sharding module, fully utilizing the parallelism of each account's dedicated channel while avoiding duplicate data collection across shard groups. Furthermore, for tasks where the shard account group identifier does not indicate the dispatch account list, i.e., public tasks without a specified list, a better channel is searched among all available accounts in the region.

[0048] See Figure 1 In one possible implementation of the scheduling method shown, the scheduling method further includes: when the designated account identifier indicates the designated power trading account and the parallel processing capacity of the shard group to which the designated power trading account belongs does not meet the disclosure window timeliness requirement, the account selection module selects a shard group that meets the capacity constraint from the regional account pool associated with the designated power trading account and enqueues it. This supports capacity-aware scheduling for large tasks. When a task specifies an account but the parallel processing capacity of its shard group (e.g., the number of accounts in the group multiplied by the throughput of a single account) is insufficient to complete within the disclosure window, it can be downgraded to an unspecified account task, and the selection module selects the shard group with the lowest occupancy and that meets the capacity constraint from the regional account pool and enqueues it.

[0049] See Figure 1 In one possible implementation of the scheduling method shown, the scheduling method further includes: when the designated account identifier indicates the designated power trading account and the designated power trading account forwards more than a threshold number of homogeneous requests associated with the power trading data collection task to the account-specific channel corresponding to the designated power trading account within a preset time period, the homogeneous requests are merged into a parent task, and then the parent task is sent to the sharding dispatch channel through a two-level task sharding module. This supports the aggregation of homogeneous requests in bursts, automatically merging structurally similar requests that flood into the same account in a short period into a multi-account sharded parent task with a list of dispatching accounts, retaining the business-specific account field for auditing, and the actual execution account is determined by the selection and re-dispatch within the sharding group.

[0050] Figure 2 This is a schematic diagram of a multi-queue topology isolated by account, provided as an embodiment of this application. Figure 2 As shown, the multi-queue topology based on account isolation includes account-specific channels A201 for account A, B203 for account B, and C203 for account C, as well as a default dispatch channel 210, a sharded dispatch channel 220, an isolation channel 230, and a real-time channel 240. This multi-queue topology with account isolation supports serial processing for single accounts to ensure a bottom line of risk control, while also supporting parallel processing across accounts to ensure overall throughput for batch homogeneous requests and multi-account collaboration.

[0051] Figure 3 This is a flowchart illustrating a method for account selection and load scoring provided in an embodiment of this application. Figure 3 As shown, the method for account selection and load scoring includes the following steps.

[0052] Step S310: No account specified, send the task to the default dispatch channel.

[0053] Step S320: Collect queue metrics for the dedicated channels of each candidate account.

[0054] Step S330: Calculate the queue depth score for each candidate account's dedicated channel.

[0055] Step S340: Read the historical time consumption estimate of each candidate account's exclusive channel.

[0056] Step S350: Select numbers in ascending order according to the double evaluation.

[0057] Step S360: Send the power trading data collection task to the selected candidate account's dedicated channel.

[0058] See Figure 3The input metrics for queue depth scoring include the number of waiting queue tasks for each candidate power trading account, the count of temporary placeholder tasks recently assigned but not yet reflected in the redistribution status within the current process, the number of tasks currently being executed, the number of delayed queue tasks, and whether currently executing tasks have been running continuously for more than a timeout threshold (e.g., 3 minutes). By assigning weights to the input metrics and then weighting and summing them, a unified scoring formula can be used to give each candidate power trading account a queue depth score. This queue depth score reflects the task execution and queue waiting status among each candidate power trading account. Furthermore, the queue depth score reflects the current status of a specific candidate power trading account. Based on the queue depth score, historical time consumption estimates are also incorporated; that is, by referring to historical data, the sum of the existing queue occupancy time for each candidate power trading account and the estimated time consumption of the currently pending tasks is further estimated as the total expected occupancy time. In some examples, historical time consumption estimation can refer to data including the remaining execution time of the current task, the sum of the estimated execution times of sampled tasks in the waiting queue, the recent delayed task consumption, the consumption of tasks already assigned but not yet enqueued, and the estimated execution time of the currently pending task itself. Here, the estimated execution time of the currently pending task is preferentially read from the historical average execution time matched by the same task name, category, and account dimension. If historical data is lacking, a rollback operation can be performed according to the default value of the task category (e.g., the default execution time of daily report tasks is 120 seconds, and the execution time of monthly report tasks is 300 seconds) or the designed global minimum value (e.g., providing a minimum of 180 seconds to complete any task category). Through the account selection module, based on the dual evaluation of queue depth score and historical time consumption estimation, the estimated start time of the current task for each candidate power trading account is estimated. This facilitates the selection of the most suitable candidate power trading account and the corresponding dedicated account channel for subsequent processes.

[0059] Continue reading Figure 3The following details how the account selection module employs a dual evaluation algorithm: queue depth score plus historical time consumption estimation. For tasks entering the default dispatch channel without a target account, the account selection module collects real-time metrics from each candidate account queue and selects a number after a dual evaluation. Specifically, the queue depth score (legacy score) can have the following input metrics (for each candidate account): number of tasks in the waiting queue (waitingCount); temporary placeholder counts of tasks recently dispatched but not yet reflected in the reallocation status within the current process (assignedCount); number of tasks currently executing (activeCount); number of tasks in the delayed queue (delayedCount); and whether the currently executing task has been running continuously for more than a preset duration, such as 3 minutes (hasLongRunningJob). Finally, the scoring formula can use a weighted evaluation algorithm. For example, the weighting factor for the number of tasks in the waiting queue (waitingCount) is 10, the weighting factor for the temporary placeholder count (assignedCount) that has just been dispatched in the current process and has not yet been reflected in the reassignment state is 8, the weighting factor for the number of tasks currently being executed (activeCount) is 5, and the weighting factor for the number of tasks in the delayed queue (delayedCount) is 3. Finally, a score from 0 to 100 can be used to indicate whether the currently executing task has been running continuously for more than a preset time, such as 3 minutes (hasLongRunningJob).

[0060] Continue reading Figure 3Regarding historical time consumption estimation, based on the queue depth score, the sum of the existing queue occupancy time for each account and the estimated time of the current pending task (estimatedOccupancyMs) is further estimated. This sum represents the total time required to complete the current task from the current moment. Historical time consumption estimation can be based on the following information, specifically including: the remaining execution time of currently active tasks (activeRemainingMs), the sum of the estimated times of sampled tasks in the waiting queue (waitingSampleSum), the occupancy of delayed tasks that will be promoted for execution soon (nearTermDelayedSum), the placeholders that have been marked as assigned but not yet enqueued (assignedInflightSum), and the estimated time of the current pending task (incomingJobDurationMs). Here, regarding the estimated time of the current pending task (incomingJobDurationMs), the historical execution time of the task name, category, and account dimension is read first. If there is no historical time, the default value of the task category (e.g., 120 seconds for daily reports, 300 seconds for monthly reports) or the global fallback value (180 seconds) is rolled back. Finally, the following formula can be used to calculate the current time plus the existing queue occupancy time (estimatedFreeAt) for the account. This represents the estimated start time for the account to execute the current task. In other words, the current time (now) is added to the sum of the existing queue occupancy time and the estimated time of the currently pending task (estimatedOccupancyMs), and then the estimated time of the currently pending task (incomingJobDurationMs) is subtracted. This allows for a real-time estimate of the time when the current account will begin executing the current task, providing a basis for subsequent processes. The formula can be used as a reference: estimatedFreeAt = now + estimatedOccupancyMs - incomingJobDurationMs.

[0061] Finally, the account selection module determines the order priority of account selection according to specific rules: (1) Exclude accounts marked as temporarily unavailable; (2) The actual number selection is based on the estimated total occupancy time (estimatedOccupancyMs) in ascending order; if there are ties, the queue depth score (legacy score) is used in ascending order, and the earlier last selection time (lastPickedAt) and account name are used as references. Therefore, the order priority of account selection includes estimated occupancy time, queue score, last selection time and account name. After selecting an account, the temporary occupancy status of the account is updated first, that is, the placeholder in the update process is updated, and then the task is forwarded to the corresponding account-specific channel and the configuration is updated to avoid multiple tasks selecting the same account consecutively before the queue status is updated. In addition, after selecting an account and forwarding the task, the task is marked as completed on the default dispatch channel side to avoid duplicate consumption. In contrast, if the task payload carries a list of dispatch accounts, the candidate accounts are limited to this list to ensure that the selected number and subsequent rebalancing and sibling reassignment are in the same shard group.

[0062] Figure 4 This is a flowchart illustrating a two-level task sharding and output aggregation method provided in an embodiment of this application. Figure 4 As shown, the two-level task sharding and output aggregation method includes the following steps.

[0063] Step S410: The parent task enters the fragmentation dispatch channel.

[0064] Step S420: First-level account-level sharding, creating sub-tasks for each account's dedicated channel.

[0065] Step S430: Second-level request item sharding, using a mapping algorithm to make request item sets under different shard numbers mutually exclusive.

[0066] Step S440: The output of each fragment sequence number is written locally.

[0067] Step S450: Once all sharding is complete and the files are in order, acquire the distributed lock, merge them by sequence number, and then upload them.

[0068] See Figure 4 After two-level sharding, requests that were originally completed individually by the parent task are distributed to various subtasks, which are then completed together by the subtasks of different dispatching accounts under the same parent task. Therefore, the subtasks of different dispatching accounts under the same parent task can be considered a shard group, and the sharding progress of the parent task is updated by monitoring the progress of each of these subtasks. For example, the parent task is responsible for maintaining a list of requests dispatched after two-level sharding, and using algorithms such as polling to wait for all subtasks to complete before updating the sharding progress percentage. The parent task enters a success state after all subtasks have successfully completed.

[0069] Figure 5 This diagram illustrates multiple paths for dynamic rebalancing and sibling account reassignment, as provided in an embodiment of this application. Figure 5 As shown, the dynamic rebalancing and sibling account reassignment have multiple paths, including path A510 (intra-group rebalancing), path B520 (account unavailable), and path C530 (sibling reassignment). Thus, three different paths are defined. Path A510 (intra-group rebalancing) corresponds to situations where migration within a shard group times out, such as when the wait time exceeds the preset 60-second duration and the number of migrations does not exceed the maximum (e.g., 2 times). Path B520 (account unavailable) corresponds to situations such as account bans; by migrating to the least occupied available account, long wait times for public tasks are avoided. Path C530 (sibling reassignment) corresponds to prioritizing untried sibling accounts within the same shard group after a shard subtask fails, helping to maintain the efficiency of large-scale parallel processing.

[0070] Continue reading Figure 5 Regarding dynamic rebalancing and sibling reassignment, a background timer (e.g., every 30 seconds by default) scans the waiting queues of each account channel for tasks forwarded by the default dispatch channel and marked as eligible for dynamic rebalancing. The dynamic rebalancing process may include: re-delivering equivalent tasks to the target account channel during migration; retaining the original task payload and retry policy after migration; deleting waiting tasks in the original channel after a new task enters the target channel; updating the cumulative migration count of tasks; marking unavailable accounts with an expiration date, which is automatically lifted upon expiration; and maintaining the original shard sequence number and request item mapping after shard subtask reassignment, only changing the execution account to avoid duplicate data collection and mapping changes.

[0071] See Figures 1 to 5 In one possible implementation, the scheduling method for power trading data acquisition tasks supports dual-threshold cross-group rebalancing. For example, in addition to the existing 60-second waiting time and 30-second idle time on a machine within a shard group, a higher waiting threshold is added. Furthermore, if a timeout still occurs after intra-group migration, cross-shard group reassignment can be allowed under limited conditions.

[0072] Figure 6This is a schematic diagram of a computer device 600 provided in an embodiment of this application. The computer device 600 includes one or more processors 610, a communication interface 620, and a memory 630. The processors 610, the communication interface 620, and the memory 630 are interconnected via a bus 640. Optionally, the computer device 600 may further include an input / output interface 650, which is connected to input / output devices for receiving user-set parameters, etc. The computer device 600 can be used to implement some or all of the functions of the device embodiment or system embodiment in the above-described embodiments of this application; the processor 610 can also be used to implement some or all of the operation steps of the method embodiment in the above-described embodiments of this application. For example, the specific implementation of various operations performed by the computer device 600 can be referred to the specific details in the above embodiments, such as the processor 610 being used to execute some or all of the steps or operations in the above-described method embodiments. For example, in the embodiments of this application, the computer device 600 can be used to implement some or all of the functions of one or more components in the above-described device embodiments. In addition, the communication interface 620 can be used specifically for communication functions necessary to implement the functions of these devices and components, and the processor 610 can be used specifically for processing functions necessary to implement the functions of these devices and components.

[0073] It should be understood that, Figure 6 The computer device 600 may include one or more processors 610, and the multiple processors 610 may cooperate to provide processing power in a parallel connection mode, a serial connection mode, a serial-parallel connection mode, or an arbitrary connection mode; or the multiple processors 610 may form a processor sequence or a processor array; or the multiple processors 610 may be divided into a main processor and an auxiliary processor; or the multiple processors 610 may have different architectures, such as adopting a heterogeneous computing architecture. Furthermore, Figure 6 The structural and functional descriptions of the computer device 600 shown are exemplary and non-limiting. In some exemplary embodiments, the computer device 600 may include... Figure 6 The diagram shows more or fewer components, or combinations of some components, or splitting of some components, or different arrangements of components.

[0074] The processor 610 can have various specific implementations. For example, the processor 610 may include one or more combinations of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a tensor processing unit (TPU), or a data processing unit (DPU), etc. This application embodiment does not impose specific limitations. The processor 610 can also be a single-core processor or a multi-core processor. The processor 610 can be a combination of a CPU and hardware chips. The aforementioned hardware chips can be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLDs can be complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), or any combination thereof. The processor 610 can also be implemented solely using logic devices with built-in processing logic, such as FPGAs or digital signal processors (DSPs). The communication interface 620 can be a wired interface or a wireless interface, used to communicate with other modules or devices. The wired interface can be an Ethernet interface, a local interconnect network (LIN), etc., and the wireless interface can be a cellular network interface or a wireless LAN interface, etc.

[0075] Memory 630 may be non-volatile memory, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Memory 630 may also be volatile memory, which may be random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM). The memory 630 can also be used to store program code and data, so that the processor 610 can call the program code stored in the memory 630 to execute some or all of the operation steps in the above method embodiments, or to execute the corresponding functions in the above device embodiments. Furthermore, the computer device 600 may include, compared to... Figure 6 The number of components displayed may be more or less, or there may be different component configurations.

[0076] Bus 640 can be a Peripheral Component Interconnect Express (PCIe) bus, or an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. Bus 640 can be divided into address bus, data bus, control bus, etc. In addition to the data bus, bus 640 can also include a power bus, control bus, and status signal bus. However, for clarity, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0077] Figure 7 This is a schematic diagram of a scheduling system for power transaction data acquisition tasks, provided as an embodiment of this application. Figure 7 As shown, the scheduling system 700 includes a multi-queue topology layer 710, an account selection layer 720, a dynamic rebalancing layer 730, a two-level task sharding layer 740, a high-availability reassignment layer 750, and an execution observation layer 760. The scheduling system is used to: through the account selection layer 720, select a corresponding queue type channel and a corresponding concurrency strategy from multiple queue type channels based on multiple identifiers of the power trading data acquisition task; and execute the scheduling and dispatch of the power trading data acquisition task using the corresponding queue type channel and the corresponding concurrency strategy. The multiple identifiers include at least a designated account identifier and a sharded account group identifier. The multiple queue type channels include at least multiple account-specific channels and sharded dispatch channels that correspond one-to-one with multiple power trading accounts. The designated account identifier is used to select a specific power trading account from the multiple power trading accounts. The sharded account group identifier is used to select a list of dispatched accounts. The list of dispatched accounts is used to select at least two power trading accounts from the multiple power trading accounts that are in the same shard group.

[0078] Continue reading Figure 7 The multi-queue topology layer 710 is used to provide a default dispatch channel, multiple account-specific channels, and sharded dispatch channels. The account selection layer 720 is used to select accounts based on queue status and historical latency. The dynamic rebalancing layer 730 is used to determine migration conditions and perform task reassignment. The two-level task sharding layer 740 is used to perform account-level sharding and request-item-level sharding. The high-availability reassignment layer 750 is used to select other accounts within the same sharding group when an account fails. The execution observation layer 760 is used to record task execution time, migration count, failure summary, and results of missed execution and delayed execution.

[0079] See Figure 7The scheduling system 700 proposes a distributed task scheduling architecture that simultaneously features single-account serial bottom line, joint number selection based on depth and time consumption, priority rebalancing within shard groups, two-level sharding, sibling reassignment, and merged upload. The execution observation layer 760 can perform a series of observation functions, including: writing the processing timestamp to the dispatch ordered set index upon task completion (e.g., by region / task dimension, with configurable retention period); matching the scheduling health module against the index records within a configurable time window based on the expected trigger list for the day, and outputting the conclusion of missed / delayed execution; recording the start and end times, channel migration counts, and failure summaries for each execution attempt, providing the management layer with a retry trajectory. The scheduling system 700 can refer to the following overall process.

[0080] (1) Triggered on a timer, the task is sent to the default dispatch channel, the specified account channel or (when there are multiple accounts) the segmented dispatch channel.

[0081] (2) The default distribution channel is selected based on the time consumed and the depth score, and then forwarded to the dedicated channel.

[0082] (3) For large tasks with multiple accounts, the distribution channels are distributed in segments in sequence, sub-tasks are created, request items within each sub-task are hashed and filtered, executed in parallel, and disks are merged and uploaded.

[0083] (4) The sub-task account is invalid, and the brother in the sub-group is reassigned.

[0084] (5) Waiting timeout, rebalance migration within the shard group or regional pool.

[0085] (6) Task completed, update processing index, optional delivery quality verification.

[0086] Figure 7 The scheduling system 700 shown has the following beneficial effects.

[0087] (1) Balance risk control and throughput: single account serialization ensures the bottom line of risk control; in-depth scoring and historical time consumption are combined for number selection and rebalancing to improve account utilization and avoid long-time tasks being piled up on the same account.

[0088] (2) Large tasks can be scaled and collected without duplication: The two-level sharding combines “grouping by account” with “mutually exclusive allocation by hash of request item”, enabling hundreds to thousands of homogeneous query items to be collected in parallel across accounts in the power trading scenario.

[0089] (3) Incorrect content in the shard group: sibling reassignment, unavailable migration, priority rebalancing within the shard group to reduce the impact of single account failure or long task blocking on the parent task.

[0090] (4) Ensure data integrity by merging all fragments: local disk partitioning and locking after all fragments are completed before uploading.

[0091] (5) Distinguishing from general message / stream scheduling schemes: For crawler task topology, transaction account isolation and request item-level fragmentation are used, rather than device address domain or network packet priority scheduling.

[0092] (6) Status observable: Segmentation progress, retry trajectory, and execution time index can be exposed through the query interface; missed executions can be detected.

[0093] See Figures 1 to 7 This application proposes a scheduling method, electronic device, medium, and system for power trading data acquisition tasks. Utilizing account queue isolation technology, account selection technology, rebalancing technology, and fragmented distribution technology, it supports application scenarios in the power trading market characterized by multiple accounts, strong risk control constraints, and time sensitivity, thus facilitating large-scale concurrent scheduling and low fault tolerance requirements. The following detailed explanation, using examples, further illustrates the industry-specific characteristics of power trading, such as strong risk control constraints and account collaboration.

[0094] (1) Example 1, intelligent number selection.

[0095] A public daily report task is triggered on a schedule, without specifying an account, and enters the default dispatch channel. The selection module estimates the queue depth score for three accounts (A, B, and C): A's queue depth score is 25 (2 waiting, 1 active), but its historical execution time is estimated at 420 seconds (this is because there are already monthly report tasks in the queue). Conversely, both B and C's queues are idle, with a historical execution time estimated at 120 seconds. Although A's queue depth score is not the highest, the system selects B in ascending order of execution time and then forwards it to B's dedicated channel for serial execution.

[0096] (2) Example 2, large task with multiple accounts (e.g., homogeneous query in power trading).

[0097] A contract disclosure task is configured with four dispatch accounts, containing 400 structurally similar "counterpartner combinations" query requests, which enter the sharding dispatch channel. The parent task creates four sub-tasks, with the sharding progress increasing from 25% to 100%. Within each sub-task, requests are filtered using a hash modulo algorithm, with approximately 100 requests executed per sub-task. After the four shards are written to disk, a merge lock is acquired, and the objects are concatenated and uploaded to object storage according to their sequence numbers. If executed serially through a single-account channel, the estimated execution time is approximately four times that of a parallel solution.

[0098] (3) Example 3, reassignment of siblings within the shard group.

[0099] When the four-part task progressed to 75%, a subtask for a certain account failed due to authentication failure. The reassignment module selected the untried sibling account E from the dispatch list of the same parent task, reassigned the subtask, and updated the allocation, keeping the original request mapping unchanged. The parent task eventually completed successfully.

[0100] (4) Example 4, waiting for rebalancing within the shard group.

[0101] In channel A of a multi-sharded task, a public task forwarded by the default dispatch channel has been waiting for 90 seconds. The estimated waiting time for A is later due to a long-running active task ahead. The rebalancing module detects that account B's estimated waiting time is more than 30 seconds earlier in the task's dispatch account list's shard group and migrates the task to channel B. The task's rebalancing count is incremented by 1, remaining within the same shard group, and the shard number remains unchanged.

[0102] The methods and devices provided in this application are based on the same inventive concept. Since the principles by which the methods and devices solve problems are similar, the embodiments, implementation methods, examples, or methods of implementation of the methods and devices can be referred to each other, and repeated details will not be repeated. This application also provides a system comprising multiple computer devices, the structure of each computer device of which can refer to the structure of the computer devices described above. The functions or operations achievable by this system can refer to the specific implementation steps in the above method embodiments and / or the specific functions described in the above device embodiments, and will not be repeated here.

[0103] This application also provides a computer-readable storage medium storing computer instructions. When these computer instructions are executed on a computer device (such as one or more processors), they can implement the method steps described in the above method embodiments. The specific implementation of the above method steps by the processor of the computer-readable storage medium can refer to the specific operations described in the above method embodiments and / or the specific functions described in the above device embodiments, and will not be repeated here.

[0104] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. This application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Embodiments of this application can be implemented wholly or partially by software, hardware, firmware, or any other combination. When implemented in software, the above embodiments can be implemented wholly or partially as a computer program product. This application can take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless network communication, microwave, etc.) means. Computer-readable storage media can be any available medium that a computer can access, or a data storage device such as a server or data center that contains one or more sets of available media. Available media can be magnetic media (such as floppy disks, hard disks, and magnetic tapes), optical media, or semiconductor media. Semiconductor media can be solid-state drives, random access memory, flash memory, read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, registers, or any other suitable form of storage medium.

[0105] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. Each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0106] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. The steps in the methods of the embodiments of this application can be adjusted in order, combined, or deleted according to actual needs; the modules in the systems of the embodiments of this application can be divided, combined, or deleted according to actual needs. If these modifications and variations of the embodiments of this application fall within the scope of the claims of this application and their equivalents, then this application also intends to include these modifications and variations.

Claims

1. A scheduling method for power trading data acquisition tasks, characterized in that, The scheduling method includes: Based on the various identifiers of the power trading data acquisition task, the corresponding queue type channel and corresponding concurrency strategy are selected from multiple queue type channels; The scheduling and dispatch of the power trading data acquisition task are performed using the corresponding queue type channel and the corresponding concurrency strategy. The multiple identifiers include at least a designated account identifier and a sharded account group identifier. The multiple queue type channels include at least multiple account-specific channels and sharded distribution channels that correspond one-to-one with multiple power trading accounts. The designated account identifier is used to select a designated power trading account among the multiple power trading accounts. The sharded account group identifier is used to select a list of distribution accounts. The list of distribution accounts is used to select at least two power trading accounts that are in the same sharded group among the multiple power trading accounts.

2. The scheduling method according to claim 1, characterized in that, The multiple account-specific channels all adopt a single-concurrency strategy, and the multiple account-specific channels are isolated from each other.

3. The scheduling method according to claim 2, characterized in that, The scheduling method further includes: When the designated account identifier indicates the designated power trading account, the power trading data collection task is sent to the account-specific channel corresponding to the designated power trading account.

4. The scheduling method according to claim 3, characterized in that, The multiple queue type channels also include a default dispatch channel, and the scheduling method further includes: When the designated account identifier does not indicate any of the multiple power trading accounts, the power trading data collection task is sent to the default dispatch channel. Then, through the account selection module, a candidate account exclusive channel is selected from the multiple account exclusive channels based on a dual evaluation of queue depth score and historical time consumption estimation, and the power trading data collection task is sent to the candidate account exclusive channel. When the segmented account group identifier indicates the distribution account list, the power trading account corresponding to the candidate account's exclusive channel is limited to the distribution account list.

5. The scheduling method according to claim 2, characterized in that, The scheduling method further includes: The two-level task sharding module determines whether the power trading data collection task is configured with the sharding account group identifier and whether the number of power trading accounts included in the distribution account list after deduplication and login availability filtering is greater than one. If so, it is determined that the power trading data collection task has triggered the sharding trigger condition. Then, the power trading data collection task is sent to the sharding distribution channel as the parent task. Through the two-level task sharding module, first-level account-level sharding is performed to create multiple subtasks associated with the parent task. Each subtask is associated with the instance identifier of the parent task and the shard number corresponding to the distribution account list. Furthermore, each subtask is associated with a power trading account included in the same shard group in the distribution account list, and the subtasks are associated with each other without repetition.

6. The scheduling method according to claim 5, characterized in that, The scheduling method further includes: The two-level task sharding module performs second-level request item-level sharding to deduplicate and remap the shardable request items of the multiple subtasks, while ensuring that the same request item is mapped to the same shard number and that request item sets under different shard numbers are mutually exclusive.

7. The scheduling method according to claim 6, characterized in that, The scheduling method further includes: The multiple subtasks are sent separately to the account-specific channels corresponding to their respective associated power trading accounts. The sharding progress of the parent task is updated by detecting the progress of each of the multiple subtasks. The outputs of each of the multiple subtasks are written to the local sharding buffer. After the sharding progress of the parent task is completed, the outputs of each of the multiple subtasks stored in the local sharding buffer are merged and uploaded.

8. The scheduling method according to claim 7, characterized in that, The scheduling method further includes: When the power trading account associated with any of the multiple subtasks is unavailable, an untried power trading account is selected from the list of dispatch accounts in the same shard group of the parent task for task reassignment.

9. The scheduling method according to claim 4, characterized in that, The multiple identifiers also include a rebalancing identifier, and the scheduling method further includes: The dynamic rebalancing module periodically scans the waiting queues of the multiple account-specific channels to determine if any tasks meet the migration triggering conditions. When a task is identified as eligible for migration, a task migration and rebalancing operation is performed based on its rebalancing identifier and shard account group identifier. The task is forwarded through the default dispatch channel. The migration triggering conditions include the task's waiting time exceeding a minimum waiting threshold and the migration benefit of migrating the task from the current power trading account to a candidate power trading account exceeding a migration benefit threshold. Specifically, based on the rebalancing identifier of the task to be migrated and the shard account group identifier, the task migration and rebalancing operation is performed, including: When the rebalancing identifier allows migration and the shard account group identifier indicates the list of dispatch accounts, an available power trading account is selected from the list of dispatch accounts in the same shard group for task reassignment. When the rebalancing identifier allows migration and the shard account group identifier does not indicate the assignment account list, an available power trading account is selected from the plurality of power trading accounts for task reassignment.

10. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the scheduling method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer device, cause the computer device to perform the scheduling method according to any one of claims 1 to 9.

12. A scheduling system for power trading data acquisition tasks, characterized in that, The scheduling system includes a multi-queue topology layer, an account selection layer, a dynamic rebalancing layer, a two-level task sharding layer, a high-availability reassignment layer, and an execution observation layer. The scheduling system is used for: Through the account selection layer, based on multiple identifiers of the power trading data collection task, the corresponding queue type channel and corresponding concurrency strategy are selected from multiple queue type channels; The scheduling and dispatch of the power trading data acquisition task are performed using the corresponding queue type channel and the corresponding concurrency strategy. The various identifiers include at least a designated account identifier and a sharded account group identifier. The various queue type channels include at least multiple account-specific channels corresponding one-to-one with multiple power trading accounts and sharded distribution channels. The designated account identifier is used to select a specific power trading account from the multiple power trading accounts. The sharded account group identifier is used to select a list of distribution accounts. The list of distribution accounts is used to select at least two power trading accounts from the multiple power trading accounts that belong to the same sharded group. The multi-queue topology layer provides a default dispatch channel, multiple account-specific channels, and a sharded dispatch channel. The account selection layer selects accounts based on queue status and historical latency. The dynamic rebalancing layer determines migration conditions and reassigns tasks. The two-level task sharding layer performs account-level and request-item-level sharding. The high-availability reassignment layer selects other accounts within the same shard group when an account fails. The execution observation layer records task execution time, migration count, failure summary, and results of missed or delayed executions.

Images