A data processing operation scheduling management method and system based on deep reinforcement learning

Abstract

The application discloses a data processing operation scheduling management method and system based on deep reinforcement learning, relates to the technical field of ETL processing scheduling management of a large amount of business data in the field of financial technology, and comprises the following steps: obtaining data processing operation task data based on ETL operation scheduling configuration parameters, wherein the data processing operation task data comprises task basic identification data, task processing requirement data and task constraint condition data; executing ETL operation based on the data processing operation task data, and synchronously collecting system hardware resource state data in the execution process; generating a state information vector based on the data processing operation task data and corresponding system hardware resource state data; inputting the obtained state information vector into a deep deterministic policy gradient (DDPG) based reinforcement learning model to generate a decision result containing an ETL operation scheduling sequence, a resource allocation scheme and a concurrent execution strategy, and driving an unknown ETL data processing flow based on the decision result. The application can efficiently complete data processing operation.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and particularly to the field of ETL (Extract-Transform-Load) processing, scheduling, and management technology for large-scale business data in the financial technology field. More specifically, it relates to a job scheduling management device and method based on a deep reinforcement learning model (DDPG, Deep Deterministic Policy Gradient) to solve the problems of dynamic resource allocation, efficient scheduling, and real-time monitoring optimization of multi-priority and multi-dependency ETL jobs in financial business scenarios. Background Technology

[0002] With the rapid evolution of fintech, business data is growing explosively, creating an urgent need for integrating heterogeneous data sources. Extract-Transform-Load (ETL), as a core component of data warehouse construction and data service support, directly impacts the efficiency and stability of its scheduling and management, affecting real-time decision-making, end-of-day clearing, and regulatory data reporting capabilities in financial businesses. It is a key support for the digital transformation of financial institutions. Broadly defined data processing operations management also encompasses tasks such as data cleaning and format conversion, and its efficient management is crucial for business continuity.

[0003] Current mainstream solutions in data processing and management rely on traditional models, which present significant technical bottlenecks. In models dependent on human experience, administrators must manually assign tasks, schedule resources, and monitor operations. This is not only time-consuming and labor-intensive, but also carries high uncertainty due to human judgment bias and monitoring lag, easily leading to delays in critical tasks or data loss. Compared to human experience, rule engines offer greater operability, but their static nature still presents multiple challenges in complex scenarios. Currently, existing technologies mainly suffer from the following problems: First, the task characteristics are not fully considered. The rule engine only evaluates priorities based on static standards such as submission time and preset type, which cannot take into account the timeliness requirements of tasks, computing resource requirements, and inter-task dependencies, easily leading to resource mismatch. Second, resource scheduling efficiency is low. It cannot perceive the hardware and software status of the computing cluster in real time (such as CPU utilization and memory usage), making it difficult to cope with load fluctuations, resulting in low resource utilization and extended task processing time. Third, the monitoring mechanism is imperfect. It only records the "success / failure" status of tasks and lacks multi-dimensional data such as processing scale, resource consumption trends, and operation logs, making it impossible to accurately locate the root cause of the problem and forming a missing "monitoring-feedback-optimization" closed loop. Fourth, reliance on human experience leads to low efficiency and high uncertainty, and the static nature of the rule engine makes it difficult to adapt to complex dependency scenarios. The aforementioned problems directly constrain the timeliness and stability of core financial operations, necessitating a new management solution with intelligent decision-making capabilities. Summary of the Invention In view of this, the present invention provides a data processing job management method and system based on deep reinforcement learning technology, which is used to at least solve some of the technical problems in the background technology, specifically solving problems such as resource perception lag, difficulty in fault location, and inefficient task allocation in data processing job scheduling, adapting to large-scale data ETL scenarios in the financial technology field, and realizing intelligent scheduling and management of the entire job process.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: On the one hand, this invention discloses a data processing job scheduling and management method based on deep reinforcement learning, including the following steps: Based on the ETL job scheduling configuration parameters, obtain the data processing job task data under the ETL job scheduling configuration parameters, including task basic identification data, task processing requirement data, and task constraint data; Based on the data processing task data, an ETL job is executed, and system hardware resource status data is collected synchronously during the execution process; Based on the data processing task data and the corresponding system hardware resource status data, a status information vector is generated; The obtained state information vector is input into a reinforcement learning model based on Deep Deterministic Policy Gradient (DDPG) to generate a decision result that includes an ETL job scheduling sequence, resource allocation scheme, and concurrent execution strategy. The execution of the unknown ETL data processing flow is then driven based on this decision result.

[0005] Furthermore, the ETL job scheduling configuration parameters specifically include: The core content, scheduling priority, dependencies, and timed startup rules of ETL jobs; The core components of the ETL job include data extraction rules, data transformation logic, and data loading targets. The scheduling priority includes multiple task execution priorities divided according to the degree of impact of the job on the business and the timeliness requirements of data; The dependencies include pre-dependencies and post-dependencies between jobs; The scheduled startup rules include the task startup time, execution cycle, and task triggering conditions.

[0006] Furthermore, in the step of generating the state information vector, the obtained state information vector includes: Task status: Comprehensive resource requirement index, execution urgency, dependency satisfaction, historical execution success rate, data volume level, processing complexity, first execution identifier, timeout risk level; System resource status: Cluster resource idle index, number of idle CPU cores, amount of idle memory, disk I / O idle bandwidth, number of currently executing tasks, and recent task failure rate; Inter-task relationship status: resource contention level, number of tasks to be executed in the same business domain, number of tasks to be executed with the same priority, number of downstream tasks, frequency of new tasks, and average urgency of downstream tasks.

[0007] Furthermore, the obtained state information vector is input into the reinforcement learning model step based on Deep Deterministic Policy Gradient (DDPG). The reinforcement learning model includes an Actor network and a Critic network, wherein: The Actor network adopts a fully connected neural network structure. The input layer dimension is consistent with the state vector dimension, and the output layer outputs decision parameters including task execution order weights, expected completion time coefficients, and concurrency coefficients. The Critic network adopts a fully connected neural network structure. The input layer is a concatenation vector of the state vector and the action output by the Actor network, and the output layer outputs the action value evaluation.

[0008] Furthermore, the reinforcement learning model training process specifically includes: An experience replay pool is used to store interactive samples, and a target network soft update mechanism is used to improve training stability.

[0009] Furthermore, a decision result is generated, including the ETL job scheduling sequence, resource allocation scheme, and concurrent execution strategy, specifically including: Task execution order by order weight Sort, where ; Tasks The comprehensive index of resource demand and the urgency of execution; Estimated completion time of the task The calculation formula is: ; in For the task The base execution time is obtained from historical data statistics. For the task The comprehensive index of resource demand, This is a comprehensive index of cluster resource idleness. The formula for calculating the number of concurrent jobs K is: ; in This represents the maximum concurrency of the cluster. This is a comprehensive index of cluster resource idleness. This refers to the degree of resource competition between tasks.

[0010] Another aspect of the present invention discloses a data processing job scheduling and management system based on deep reinforcement learning, comprising: Parameter configuration module: Used to configure ETL job scheduling parameters; Data processing task data acquisition module: used to acquire data processing task data under the ETL job scheduling configuration parameters based on the ETL job scheduling configuration parameters, including task basic identification data, task processing requirement data, and task constraint data; System hardware resource status data acquisition module: used to execute ETL jobs based on the data processing job task data, and synchronously acquire system hardware resource status data during job execution; Status information vector generation module: used to generate status information vectors based on the data processing task data and corresponding system hardware resource status data; Job scheduling decision module: It processes the obtained state information through a reinforcement learning model based on deep deterministic policy gradient (DDPG) to generate decision results that include ETL job scheduling sequences, resource allocation schemes and concurrent execution strategies, and drives the execution of unknown ETL data processing flow based on the decision results.

[0011] Preferably, the system further includes a data preprocessing module for preprocessing the data processing task data and the system hardware resource status data respectively.

[0012] Preferably, in the job execution scheduling decision module, the reinforcement learning model includes an Actor network and a Critic network, wherein: The Actor network adopts a fully connected neural network structure. The input layer dimension is consistent with the state vector dimension, and the output layer outputs decision parameters including task execution order weights, expected completion time coefficients, and concurrency coefficients. The Critic network adopts a fully connected neural network structure. The input layer is a concatenation vector of the state vector and the action output by the Actor network, and the output layer outputs the action value evaluation.

[0013] Preferably, the job execution scheduling decision module generates decision results that include ETL job scheduling sequences, resource allocation schemes, and concurrent execution strategies, specifically including: Task execution order by order weight Sort, where ; Tasks The comprehensive index of resource demand and the urgency of execution; Estimated completion time of the task The calculation formula is: ; in For the task The base execution time is obtained from historical data statistics. For the task The comprehensive index of resource demand, This is a comprehensive index of cluster resource idleness. The formula for calculating the number of concurrent jobs K is: ; 1.1 of which This represents the maximum concurrency of the cluster. This is a comprehensive index of cluster resource idleness. This refers to the degree of resource competition between tasks.

[0014] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a data processing job management method and system based on deep reinforcement learning technology, which has the following beneficial effects: This invention deeply integrates deep reinforcement learning technology to optimize data processing job management scenarios. Based on a thorough consideration of pre-defined hardware resource utilization and task dependencies, it employs deep reinforcement learning methods to conduct in-depth analysis and feature extraction of key parameters such as resource utilization, execution time, and data volume involved in daily batch processing tasks. A task priority evaluation model constructed using deep reinforcement learning methods accurately assesses the priority of data processing jobs and dynamically adjusts task allocation strategies within the task and resource environment. This method not only effectively improves system resource utilization and shortens overall data processing time but also enhances system stability and efficiently completes data processing jobs. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 A schematic diagram of the basic data processing flow provided for embodiments of the present invention.

[0017] Figure 2 This is a schematic diagram of the model optimization and data processing scheme provided in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the overall data processing scheme provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention constructs a policy network based on reinforcement learning. It takes the current state of a Directed Acyclic Graph (DAG) task graph (including task execution progress and resource allocation) as input and outputs an action, such as adjusting the task execution order or reallocating resources. The policy network learns the optimal policy by continuously interacting with the environment (i.e., executing tasks and observing the results and environmental feedback) to maximize a certain reward function. For example, the reward function is defined as the time taken to complete a task and the total time taken to complete the overall task, outputting an optimized data processing scheme.

[0021] This invention allocates tasks based on task priority and the real-time resource status of computing nodes. The algorithm learns the optimal task allocation strategy by continuously interacting with the existing environment. It can react quickly to dynamic changes in computing node resources, allocating tasks to nodes with high resource utilization and low idle capacity. Compared to traditional static allocation methods, this invention's dynamic allocation mechanism can adapt to complex and changing task and resource environments, improving resource utilization and task processing efficiency.

[0022] Example 1 refer to Figures 1-3 Embodiment 1 of the present invention discloses a data processing job scheduling and management method based on deep reinforcement learning, comprising the following steps: Acquire data processing task data and system hardware resource status data during the execution of processing tasks.

[0023] Data acquisition for data processing tasks. The data acquisition unit, deployed on the system scheduling node, performs the data acquisition operation for data processing tasks and establishes a communication connection with the data task submission terminal. The acquisition triggering mechanism includes timed triggering and event triggering: timed triggering actively retrieves data task data at preset time intervals, which can be configured within a certain range; event triggering occurs when the data task submission terminal completes task submission and sends a "task addition notification," at which point the data acquisition unit responds in real time and performs the acquisition operation. The data task acquisition unit acquires all data processing task data in the system that is pending, in execution, or paused through timed and event-triggered mechanisms. After integrity and legality verification, the data is stored in a dedicated task data database and synchronized to the job scheduling decision module cache unit.

[0024] Data Acquisition Scope and Content Definition. The data acquisition scope for data processing tasks covers all data processing tasks within the system that are pending, in progress, or paused. The core content includes: Basic task identification data includes: unique task ID, task name, business domain to which the task belongs, task creator ID, and task creation timestamp; Task processing requirement data: includes data input source information, data output target information, and processing logic description, wherein the database authentication information is stored using asymmetric encryption; Task constraint data includes task execution deadline, task priority level, task dependency data, and estimated task resource requirements. The task priority level is coded from 1 to 5, with level 1 being the highest priority and level 5 being the lowest priority.

[0025] Execute processing tasks and simultaneously collect system hardware resource status data during the operation.

[0026] The processing steps for data processing task data and system hardware resource status data are divided into two main stages: data preprocessing and status information construction. The specific operations for each stage are as follows: Data preprocessing aims to eliminate data noise, standardize data formats, and integrate multi-source data. It specifically includes three steps: data cleaning, data standardization, and data association and fusion. The detailed operational procedures for each step are as follows: The data preprocessing unit performs preprocessing operations on the task data after it has been entered into the database. The steps include: Data cleaning: Data cleaning addresses outliers, missing values, and logical conflicts in task data to ensure data quality meets the computational requirements of subsequent scheduling. An outlier removal algorithm is used to handle outliers in the task resource requirement estimation data. For missing fields after outlier removal, the average resource requirement for the same task is used to fill in the missing values. For missing core fields in the task data, default values ​​or derived values ​​are used. Circular dependencies in the task dependency data are marked as "dependency anomalies" and pushed to operations personnel for processing. Data standardization transforms task data with different dimensions and value ranges into a unified format, ensuring that all indicators can participate in comprehensive calculations. Numerical indicators such as estimated task resource requirements (number of CPU cores, memory usage, and the time difference between the execution deadline and the current time) and system resource status data (CPU idle percentage, memory idle percentage) are processed using the Min-Max normalization algorithm, with the following formula: .in, These are the original values. This is the minimum value of the corresponding indicator for the same type of task within the past 30 days of the system's history. This represents the maximum value of the corresponding indicator; this formula maps all numerical indicators to the [0,1] interval, eliminating dimensional differences.

[0027] Data association and fusion involves linking multi-source task data stored in separate tables, including basic task identifier data, processing requirement data, and constraint data, into a single complete task data record through primary keys. This ensures data integrity and callability, and the associated task data records are then uniformly converted into the same format.

[0028] State information construction. Based on the preprocessed complete task data records and combined with the system's real-time resource status data, a fixed-dimensional state information vector is constructed. This vector needs to comprehensively reflect "task attributes, system resource status, and inter-task relationships". The specific construction steps and dimension definitions are as follows: The status information dimension, based on preprocessed task data and combined with the system's real-time resource status, constructs the status information of the data processing operation. The status information vector contains three main categories and a total of 20 dimensions. The status information is a vector with fixed dimensions, which can be expanded according to the number of tasks and the cluster size. The definitions of each dimension of the vector are as follows: Task's own state dimensions (8 dimensions) (Comprehensive Task Resource Requirement Index): This index is calculated based on the estimated CPU, memory, and disk I / O requirements of a comprehensive task, weighted by business requirements. The formula is as follows: ; Among them, the weighting coefficient ; All data are normalized resource requirements data, sourced from preprocessed task constraint data.

[0029] (Task Execution Urgency): Calculated based on the time difference between the task execution deadline and the current time, using the following formula: ; in, ; and These are the minimum and maximum time differences within the past 30 days, respectively. The data source is the preprocessed task constraint data and the current system time.

[0030] (Task Dependency Satisfaction): Calculated based on the percentage of completed upstream tasks, using the following formula: ; in, The number of upstream tasks completed. This represents the total number of upstream tasks. The data source is the preprocessed task constraint data and the task status data from the job operation monitoring module.

[0031] (Task historical execution success rate): Calculated based on the last 30 task execution records, using the following formula: ; in, For the number of successes, This represents the total number of executions in the last 30 times, with data sourced from the system's historical task execution logs.

[0032] (Task Data Volume Level): Divided into 1-5 different levels according to the amount of task input data, specifically: Level 1 < 1GB, Level 2 1-10GB, Level 3 10-50GB, Level 4 50-100GB, Level 5 > 100GB; The data source is the pre-processed task processing requirement data (the file size or database table capacity of the data input source).

[0033] (Task processing complexity): Classified into levels according to the complexity of processing logic (levels 1-3, level 1 is simple SQL query, level 2 is multi-table join / data transformation, level 3 is algorithm model call / complex calculation); the data source is preprocessed task processing requirement data (processing logic description).

[0034] (Task first execution identifier): The first execution of a task is marked as 1, and subsequent executions are marked as 0; the data source is the system's historical task execution log.

[0035] (Task Timeout Risk Level): Calculated based on a combination of historical timeout probabilities of similar tasks and the urgency of the current task (Levels 1-3, Level 1: No timeout risk, Level 2: Low risk, Level 3: High risk); data source is the system's historical task execution logs and... (Urgency level).

[0036] System resource status dimensions (6 dimensions) (Cluster Resource Idle Index): This index combines the idle percentages of CPU, memory, and network bandwidth. The formula is as follows: ; Among them, coefficient , The data represents the real-time idle percentage, and the data source is the real-time resource data from the system status information acquisition module.

[0037] (CPU Idle Cores): The number of CPU cores currently idle in the cluster (normalized value); data source is the system status information acquisition module.

[0038] (Free memory): The current free memory capacity of the cluster (normalized value); the data source is the system status information collection module.

[0039] (Disk I / O Idle Bandwidth): The current idle disk read / write bandwidth of the cluster (normalized value); the data source is the system status information acquisition module.

[0040] (Current number of tasks in execution): The total number of tasks currently being executed in the cluster (normalized value); data source is the job execution monitoring module.

[0041] (Recent Task Failure Rate): The percentage of task execution failures in the past 5 minutes out of the total number of executions; data source: job operation monitoring module.

[0042] Inter-task relationship state dimension (6 dimensions) (Inter-task resource contention): Calculated based on the number of tasks competing with the current task for core resources, using the following formula: ; in, The number of tasks competing for resources. This represents the total number of active tasks; the data source is the pre-processed task processing requirements data and the job operation monitoring module.

[0043] (Number of tasks to be executed in the same business domain): The number of tasks to be executed in the same business domain as the current task (normalized value); the data source is the preprocessed task basic identification data and the job operation monitoring module.

[0044] (Number of tasks to be executed with the same priority): The number of tasks to be executed with the same priority as the current task (normalized value); the data source is the preprocessed task constraint data and the job operation monitoring module.

[0045] (Number of downstream tasks): The total number of downstream tasks that depend on the current task; the data source is the preprocessed task constraint data (reverse lookup based on task ID).

[0046] (Frequency of New Tasks): The number of new tasks to be executed by the system in the past 10 minutes (normalized value); data source is the job operation monitoring module.

[0047] (Average urgency of downstream tasks): The urgency of all downstream tasks. (Execution urgency) Average value; data source is the calculation result of the status information of downstream tasks.

[0048] State information vector generation and updating, vector assembly, according to "task's own state dimension ( → System resource status dimension ( → Inter-task relationship state dimension ( Following a fixed order, the values ​​from 20 dimensions are concatenated into a state information vector. .

[0049] Real-time updates and input transmission. The update frequency of the state information vector is synchronized with the system resource status acquisition frequency and task status change frequency. When the system resource status or task status changes, the state information vector is triggered to be recalculated and updated, ensuring that the vector can reflect the current system and task status in real time. (Synchronized updated state information vector) The data is transmitted in real time to the job scheduling decision module in step two via the API interface, serving as the core input to the DDPG-based reinforcement learning model. This provides data support for the generation of subsequent decision-making results.

[0050] A reinforcement learning model based on DDPG is used to obtain a decision based on state information. The DDPG-based reinforcement learning model includes an Actor network and a Critic network. The model construction includes: Network structure design: The Actor network adopts a fully connected neural network, with the input layer dimension consistent with the state information vector dimension, and the output layer dimension matching the decision variable dimension; the Critic network also adopts a fully connected neural network, with the input layer being a vector concatenated from the state information vector and the action vector output by the Actor network, and the output layer being a single neuron that outputs the action value; Network parameter initialization: The weight parameters of the Actor network and the Critic network are initialized using the Xavier initialization method, and the bias parameters are initialized to 0; Experience replay pool construction: The experience replay pool is used to store experience samples generated by the interaction between the model and the environment. Experience samples include the current state, executed actions, immediate rewards, and the next state. The capacity of the experience replay pool is preset. A random sampling strategy is used to select samples for network training. The sampling batch size is preset. Target network soft update mechanism: Set corresponding target networks for Actor network and Critic network respectively. The target network parameters are soft-updated from the main network at a preset number of steps, and the update coefficients are preset.

[0051] After the model input and decision-generated state information vector are input into the DDPG-based reinforcement learning model, the output state information vector of the model is generated from the preceding input. As input, a decision is generated through the following steps: The Actor network receives a state information vector and calculates a preliminary decision through forward propagation. This preliminary decision includes parameters such as the execution order of data tasks, the estimated completion time of data tasks, and the number of data processing jobs that can run concurrently. The Actor network's action output formula is based on the Actor network's received state. Output action (That is, the parameterized representation of the decision result, including execution order weights, concurrency coefficients, etc.), the formula is: ; in, For Actor network mapping functions, For Actor network parameters; actions The dimensions are matched with the decision variables, as in one specific implementation:

[0052] in, For the task Execution order weights For the task The expected completion time coefficient, This is the concurrency coefficient, and The range of values ​​is determined by step one. Constraints (such as) , (This represents the maximum concurrency of the cluster).

[0053] The Critic network receives the state information vector and the initial decision output by the Actor network, and calculates the value of the output action through forward propagation to evaluate the quality of the initial decision; The model adjusts and optimizes the initial decision based on the action value output by the Critic network and a preset optimization objective. The optimization objective is to maximize long-term cumulative returns, ultimately generating the optimal decision result. The Critic network value evaluation formula is as follows: ; Among them, network reception status With action Output action value , For the Critic network value function, For Critic network parameters; The calculation needs to be associated with step one. For example, in one specific implementation:

[0054] in, As weight, For action The actual returns ensure that the value assessment is directly related to the urgency of the task, the success rate, and the resource requirements.

[0055] Actor network parameter update formula: Maximize the policy objective function Update for target The formula is: ; ; in, For experience replay pool (storage) , (The next state is generated by the state update mechanism in step one). The learning rate for the Actor network; gradient calculation needs to be based on step one. Ensure that the direction of parameter updates matches the system status and task attributes.

[0056] The Critic network parameter update formula aims to minimize the temporal difference error (TD error). The formula is: ; ; ; in, For target value, For immediate returns: The core indicators associated with step one; As a discount factor, For the target Critic network output, For the target Actor network output, For the target network parameters, The Critic network learning rate is used; the TD error calculation directly depends on the state indicators in step one, ensuring that parameter updates are strongly correlated with the actual attributes of the task.

[0057] Target network parameters Soft update using the following formula: ; ; in, To update coefficients (default) The frequency of soft updates is determined by the state update frequency in step one (e.g., one soft update of the target network is performed every 10 state information updates).

[0058] Decision results are conveyed through actions The parameter mapping yields information including the execution order of data tasks and the estimated completion time of each data task. The execution order of data tasks is sorted by task priority, urgency, and the degree of matching between resource requirements and system idle resources; and by task execution order weight. Sort, Depend on calculate: ; in, Tasks Resource demand comprehensive index, execution urgency, The larger the value, the earlier the task will be executed.

[0059] The estimated completion time of the data task is calculated using a time estimation model based on the estimated task resource requirements, the current resource allocation of the system, and the execution time of similar tasks in the past, accurate to the minute.

[0060] ; in For the task The base execution time (derived from historical data statistics). For the task The comprehensive index of resource demand, This is a comprehensive index of cluster resource idleness. The larger, The smaller, The larger the value, the longer the execution time, which aligns with the actual logic of high resource requirements and low cluster idle time.

[0061] The number of data processing jobs that can run concurrently at the same time. The K value is dynamically determined based on the total amount of idle resources in the cluster, the average resource consumption per task, and the degree of resource contention between tasks. ; in This represents the maximum concurrency of the cluster. This is a comprehensive index of cluster resource idleness. The degree of resource competition between tasks; The larger, The smaller, The larger the number of concurrent connections, the better it is to ensure that the number of concurrent connections matches the cluster resources and the level of competition for tasks.

[0062] Example 2 Embodiment 2 of the present invention discloses a data processing job scheduling and management system based on deep reinforcement learning, comprising: The system is adapted for large-scale data ETL scenarios in the fintech field. It includes four core modules: system status information acquisition, job scheduling configuration, job execution scheduling decision, and job execution monitoring, as well as two common modules: user management and system management. These modules work together to achieve full-process scheduling and management of ETL jobs. The specific functions are as follows: The system status information acquisition module is responsible for collecting cluster hardware and software information in real time. Hardware information includes CPU utilization, memory usage, disk I / O speed, network bandwidth usage, and server temperature. Software information includes operating system process status, database connection count, middleware running status, and cluster node health status. After collection, the raw information is cleaned, feature extracted, and standardized to generate a system status information vector, which is pushed to subsequent modules in real time to solve the problem of resource awareness lag. The acquisition interval is set to 10 seconds, and an alarm is triggered when the temperature exceeds the threshold. The data is finally mapped to the [0,1] interval for unified calculation standards.

[0063] The job scheduling configuration module focuses on configuring and verifying basic ETL job information, supporting the configuration of ETL job content, scheduling priority, scheduling dependencies, and scheduled startup rules. Job content requires a unique identifier, data source, data target, and processing logic. Priority uses a 1-5 level coding system. Dependency configuration includes dependency type and timeout. Scheduled startup supports Cron expressions and fixed interval modes, and retry parameters can be configured. After configuration, mandatory fields, dependency validity, and compliance checks are performed. Qualified information is stored in a financial-grade encrypted database and synchronized to the decision module to prevent parameter corruption that could lead to task anomalies.

[0064] The job operation monitoring module is responsible for collecting and recording ETL job operation data at regular intervals, including the scale of processed data, resource utilization, and operation logs. Data scale includes the amount processed, the remaining amount, and the processing rate. Resource utilization covers the percentage of CPU, memory, disk I / O, and network bandwidth. Logs are recorded in a fixed format and retained for at least one year. The module synchronizes monitoring data to the decision-making module in real time and sets multiple alarm thresholds, notifying maintenance personnel via email, SMS, and other means to address the challenge of fault localization.

[0065] The job scheduling decision module, as the core of the system, generates decisions based on an improved DDPG reinforcement learning model. It first integrates system status information, job monitoring data, and scheduling configuration parameters to form an input vector, then outputs a scheduling sequence, resource allocation scheme, and concurrent execution strategy. The scheduling sequence is sorted by priority and dependency, resource allocation specifies the proportion of server resources, and the number of concurrent jobs is dynamically adjusted according to cluster load: 10 concurrent jobs when the load is ≤60%, and 3 concurrent jobs when the load is >80%. Decision results are delivered via a secure API with a response time of no more than 1 second. In case of an anomaly during operation, a new decision is made within 30 seconds to ensure the stable execution of core tasks.

[0066] The user management module implements system access control and account management. It adopts the RBAC permission model to assign minimum necessary permissions to different roles to avoid unauthorized operations. It supports account creation, modification, disabling, and password reset, and records user operation logs to ensure traceability and meet the requirements of financial data security management.

[0067] The system management module, belonging to the system operation and maintenance field, supports core operation and maintenance control. Its functions include: configuration management supporting batch adjustment of job parameters and alarm thresholds; version management recording the system and model iteration process, supporting version rollback and one-click deployment; performance monitoring regularly generating resource utilization and job execution efficiency reports, and possessing self-healing capabilities for common faults, reducing manual intervention to ensure system stability. The data storage support module focuses on data storage in the financial industry, aiming to address the diverse and highly secure storage needs of financial data. Its core is a layered storage system: a financial-grade encrypted database stores core data, supporting real-time backup and audit logs to prevent leaks; a data warehouse stores historical ETL and statistical data for analysis; a distributed file system stores logs and large files, supporting high-concurrency read / write operations; a caching system stores frequently accessed data to improve efficiency, and a redundant disaster recovery mechanism ensures data security and recoverability. The business adaptation module focuses on fintech ETL scenarios, enhancing the system's flexibility in adapting to business needs. It includes built-in standardized ETL templates for multiple industries, supporting direct reuse and parameter adjustments; provides interfaces for users to add industry-specific data processing logic; and establishes a dynamic configuration mechanism, allowing for rapid configuration updates when policies change or business expands, without requiring core system reconstruction, thus reducing costs and improving efficiency. The basic operations and maintenance support module adopts an integrated operations and maintenance approach, providing end-to-end operations and maintenance support. Core capabilities include: the operations layer allows for batch parameter tuning and version management, mitigating the impact of new version failures on business operations; the monitoring layer displays the real-time operating status of each module and generates comprehensive reports with multi-dimensional indicators; and the fault diagnosis layer can automatically recover from common problems, reducing operations and maintenance complexity and ensuring stable operation. The performance optimization module, applied to high-demand scenarios, is dedicated to improving system performance. Optimization solutions include: dynamically allocating resources, adjusting CPU, memory, and other resources based on business load; optimizing data processing algorithms and transmission methods to improve processing efficiency; establishing a request priority mechanism to cache the results of frequently requested requests; and real-time monitoring of performance metrics, setting early warning thresholds, and proactively addressing potential problems.

[0068] This invention uses a parameter configuration method to set an upper limit for system resource utilization and to plan and adjust the resource scheduling for system data processing.

[0069] This invention employs real-time task allocation and scheduling based on deep reinforcement learning, continuously optimizing during operation. As the amount of data increases or decreases, task allocation can be performed based on task priority and the real-time resource status of computing nodes, improving resource utilization and shortening the execution time of batch tasks.

[0070] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0071] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing job scheduling and management method based on deep reinforcement learning, characterized in that, Includes the following steps: Based on the ETL job scheduling configuration parameters, obtain the data processing job task data under the ETL job scheduling configuration parameters, including task basic identification data, task processing requirement data, and task constraint data; Based on the data processing task data, an ETL job is executed, and system hardware resource status data is collected synchronously during the execution process; Based on the data processing task data and the corresponding system hardware resource status data, a status information vector is generated; The obtained state information vector is input into a reinforcement learning model based on Deep Deterministic Policy Gradient (DDPG) to generate a decision result that includes an ETL job scheduling sequence, resource allocation scheme, and concurrent execution strategy. The execution of the unknown ETL data processing flow is then driven based on this decision result.

2. The method according to claim 1, characterized in that, The ETL job scheduling configuration parameters specifically include: The core content, scheduling priority, dependencies, and timed startup rules of ETL jobs; The core components of the ETL job include data extraction rules, data transformation logic, and data loading targets. The scheduling priority includes multiple task execution priorities divided according to the degree of impact of the job on the business and the timeliness requirements of data; The dependencies include pre-dependencies and post-dependencies between jobs; The scheduled startup rules include the task startup time, execution cycle, and task triggering conditions.

3. The method according to claim 1, characterized in that, In the step of generating the state information vector, the obtained state information vector includes: Task status: Comprehensive resource requirement index, execution urgency, dependency satisfaction, historical execution success rate, data volume level, processing complexity, first execution identifier, timeout risk level; System resource status: Cluster resource idle index, number of idle CPU cores, amount of idle memory, disk I / O idle bandwidth, number of currently executing tasks, and recent task failure rate; Inter-task relationship status: resource contention level, number of tasks to be executed in the same business domain, number of tasks to be executed with the same priority, number of downstream tasks, frequency of new tasks, and average urgency of downstream tasks.

4. The method according to claim 1, characterized in that, The obtained state information vector is input into the reinforcement learning model based on Deep Deterministic Policy Gradient (DDPG) step. The reinforcement learning model includes an Actor network and a Critic network, wherein: The Actor network adopts a fully connected neural network structure. The input layer dimension is consistent with the state vector dimension, and the output layer outputs decision parameters including task execution order weights, expected completion time coefficients, and concurrency coefficients. The Critic network adopts a fully connected neural network structure. The input layer is a concatenation vector of the state vector and the action output by the Actor network, and the output layer outputs the action value evaluation.

5. The method according to claim 4, characterized in that, The reinforcement learning model training process specifically includes: An experience replay pool is used to store interactive samples, and a target network soft update mechanism is used to improve training stability.

6. The method according to claim 1, characterized in that, Generate decision results including ETL job scheduling sequences, resource allocation schemes, and concurrent execution strategies, specifically including: Task execution order by order weight Sort, where ； Tasks The comprehensive index of resource demand and the urgency of execution; Estimated completion time of the task The calculation formula is: ； in For the task The base execution time is obtained from historical data statistics. For the task The comprehensive index of resource demand, This is a comprehensive index of cluster resource idleness. The formula for calculating the number of concurrent jobs K is: ； in This represents the maximum concurrency of the cluster. This is a comprehensive index of cluster resource idleness. This refers to the degree of resource competition between tasks.

7. A data processing job scheduling and management system based on deep reinforcement learning, characterized in that, include: Parameter configuration module: Used to configure ETL job scheduling parameters; Data processing task data acquisition module: used to acquire data processing task data under the ETL job scheduling configuration parameters based on the ETL job scheduling configuration parameters, including task basic identification data, task processing requirement data, and task constraint data; System hardware resource status data acquisition module: used to execute ETL jobs based on the data processing job task data, and synchronously acquire system hardware resource status data during job execution; Status information vector generation module: used to generate status information vectors based on the data processing task data and corresponding system hardware resource status data; Job scheduling decision module: It processes the obtained state information through a reinforcement learning model based on deep deterministic policy gradient (DDPG) to generate decision results that include ETL job scheduling sequences, resource allocation schemes and concurrent execution strategies, and drives the execution of unknown ETL data processing flow based on the decision results.

8. The system according to claim 7, characterized in that, It also includes a data preprocessing module, which is used to preprocess data processing task data and system hardware resource status data respectively.

9. The system according to claim 7, characterized in that, In the job execution scheduling decision module, the reinforcement learning model includes an Actor network and a Critic network, wherein: The Actor network adopts a fully connected neural network structure. The input layer dimension is consistent with the state vector dimension, and the output layer outputs decision parameters including task execution order weights, expected completion time coefficients, and concurrency coefficients. The Critic network adopts a fully connected neural network structure. The input layer is a concatenation vector of the state vector and the action output by the Actor network, and the output layer outputs the action value evaluation.

10. The system according to claim 7, characterized in that, The job execution scheduling decision module generates decision results that include ETL job scheduling sequences, resource allocation schemes, and concurrent execution strategies, specifically including: Task execution order by order weight Sort, where ； Tasks The comprehensive index of resource demand and the urgency of execution; Estimated completion time of the task The calculation formula is: ； in For the task The base execution time is obtained from historical data statistics. For the task The comprehensive index of resource demand, This is a comprehensive index of cluster resource idleness. The formula for calculating the number of concurrent jobs K is: ； in This represents the maximum concurrency of the cluster. This is a comprehensive index of cluster resource idleness. This refers to the degree of resource competition between tasks.

Images