A data processing method, system, device and storage medium

Abstract

The application provides a data processing method, system, device and storage medium, relates to the technical field of data processing, and comprises the following steps: receiving a plurality of task creation requests sent by a business system, creating a plurality of business tasks; generating a first processing order of the plurality of business tasks in combination with the processing priorities and the association indexes of the business tasks; adjusting the first processing order according to the resource occupation amounts of the business tasks, and generating a second processing order; calculating delay risk indexes of the business tasks, adjusting the second processing order according to the delay risk indexes, and generating a target processing order; the delay risk index is used for representing the degree of influence of the processing delay of each business task on the business; and the plurality of business tasks are processed in the target processing order through a preset global task listener. The application has the technical effect that the scheduling efficiency in a complex business scenario is improved, and therefore the overall performance of the system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a data processing method, system, device and storage medium. BACKGROUND

[0002] In the current enterprise-level business process management system, the workflow engine (such as Flowable) usually needs to process a large number of concurrent task requests, and the processing order of the tasks directly affects the efficiency of business execution and resource utilization. How to efficiently process a large number of concurrent tasks has become a technical problem to be solved.

[0003] At present, static priority scheduling is usually used to manage the execution order of tasks. For example, an initial execution queue is generated by presetting task priority rules. Although this method is simple to implement and has small computational overhead, the above method cannot be dynamically adjusted according to the actual state of the system at runtime, resulting in low scheduling efficiency in complex business scenarios and affecting the overall performance of the system. SUMMARY

[0004] The present application provides a data processing method, system, device and storage medium for improving the scheduling efficiency in complex business scenarios and thus improving the overall performance of the system.

[0005] In a first aspect, the present application provides a data processing method, which comprises: receiving a plurality of task creation requests sent by a business system, creating a plurality of business tasks; obtaining the processing priority of each business task and the association index between each business task, generating a first processing order of the plurality of business tasks in combination with the processing priority and the association index; obtaining the resource occupation amount of each business task, adjusting the first processing order according to the resource occupation amount to generate a second processing order; calculating the delay risk index of each business task, adjusting the second processing order according to the delay risk index to generate a target processing order; the delay risk index is used to represent the degree of impact of the processing delay of each business task on the business; and processing the plurality of business tasks in the target processing order through a preset global task listener.

[0006] By adopting the technical scheme, a multi-dimensional task scheduling mechanism is constructed by comprehensively considering the processing priority of the business task, the correlation index between tasks, the resource occupation amount, and the delay risk index, and intelligent optimization of the business task processing order is realized. The scheme first establishes an initial order based on business importance, then ensures cooperative processing of related tasks through correlation analysis, further optimizes the processing order according to the delay risk index, and comprehensively balances multiple factors such as business priority, task correlation, system resource utilization efficiency, and business impact degree. The preset global task listener executes the optimized target processing order, significantly improves the scheduling efficiency in complex business scenarios, and thus improves the overall system performance.

[0007] Optionally, the processing priority of each business task and the correlation index between each business task are obtained, including: obtaining a preset priority of each business task and a processing rule of each business task, adjusting the preset priority according to the processing rule to obtain the processing priority of each business task; collecting a preset correlation level between each business task, and determining the correlation index between each business task according to the preset correlation level, the correlation index being positively correlated with the correlation level.

[0008] By adopting the technical scheme, the preset priority is dynamically adjusted by the processing rule, and the correlation index between tasks is quantified based on the preset correlation level, which realizes fine management of task priority and correlation, enables the system to more accurately identify the key path and dependency chain of task processing, avoids misplacement of high-correlation task processing, improves the flexibility and processing coordination efficiency of task scheduling, reduces delay caused by improper handling of dependency relationships, and makes the business process more smooth and efficient.

[0009] Optionally, the processing priority and the correlation index are combined to generate a first processing order of the plurality of business tasks, including: determining an initial processing order of the plurality of business tasks according to the processing priority; determining a pair of business tasks with a correlation index greater than a correlation threshold; adjusting the positions of the pair of business tasks with the correlation index greater than the correlation threshold to be adjacent in the initial processing order; and generating a first processing order of the plurality of business tasks based on the adjusted order.

[0010] By adopting the technical scheme, the initial processing order of the business tasks is determined first, and then the relative positions of the high-correlation task pairs are identified and adjusted, so that the task priority and correlation requirements are effectively balanced. The scheme first establishes an initial processing order according to the processing priority, then filters out the business task pairs with a correlation index greater than a correlation threshold, adjusts the positions of these high-correlation task pairs to adjacent positions in the initial order, and finally generates a first processing order. This processing mechanism ensures that high-correlation tasks can be executed in close proximity, reduces the switching cost and waiting time between tasks, while still maintaining basic respect for the overall priority, improving the coherence and efficiency of task processing, and reducing the risk of resource waste and processing delay caused by scattered related task processing.

[0011] Optionally, the adjusting the first processing order according to the resource occupation amount to generate a second processing order comprises: obtaining the total amount of system resources, calculating the ratio of the resource occupation amount of each business task to the total amount of system resources; determining high resource consumption tasks with a resource occupation rate greater than a resource threshold; reordering the high resource consumption tasks in the first processing order, and inserting tasks with a resource occupation rate less than the resource threshold between adjacent high resource consumption tasks; generating a second processing order based on the adjusted order.

[0012] By adopting the technical scheme, by identifying high resource consumption tasks and optimizing their distribution in the processing sequence, the problem of system resource bottleneck caused by continuous execution of resource-intensive tasks is effectively solved. The scheme first calculates the resource occupation rate of the tasks, identifies high resource consumption tasks with a resource occupation rate greater than a resource threshold, and then adjusts the first processing order by inserting tasks with a low resource occupation rate between adjacent high resource consumption tasks to form a second processing order. This resource balancing type of sorting strategy avoids the centralized execution of high resource consumption tasks, makes the system resource load more balanced, prevents the risk of system resource instantaneous tension or depletion, improves the overall throughput capacity and stability of the system, and reduces the task processing delay caused by resource competition.

[0013] Optionally, the calculating the delay risk index of each business task comprises: obtaining a preset business impact level of each business task, the business impact level being used to represent the influence degree of business task delay on the business process; obtaining the remaining processing time limit of each business task, and calculating the delay risk index of each business task according to the business impact level and the remaining processing time limit, wherein the business impact level and the delay risk index are positively correlated, and the remaining processing time limit and the delay risk index are inversely correlated.

[0014] By adopting the technical scheme, a dynamic delay risk assessment mechanism is established by comprehensively considering the business impact level and the remaining processing time limit of the business task. The preset business impact level of the scheme is obtained, which reflects the influence degree of task delay on the business process, and the remaining processing time limit is obtained, and the delay risk index is calculated accordingly, so that the business impact level is positively correlated with the delay risk index, and the remaining processing time limit is inversely correlated with the delay risk index. This mechanism enables the system to accurately identify key tasks that have high business impact and face urgent time constraints, providing a scientific basis for subsequent processing sequence adjustment and effectively preventing serious business impact caused by processing delay of high-value business tasks.

[0015] Optionally, the adjusting the second processing sequence according to the delay risk index to generate a target processing sequence comprises: arranging the business tasks in the second processing sequence in descending order of delay risk index to generate an initial risk sequence; identifying an urgent task whose delay risk index is greater than a risk threshold in the initial risk sequence; and advancing the execution position of the urgent task in the second processing sequence to generate a target processing sequence.

[0016] By adopting the technical scheme, the business tasks in the second processing sequence are arranged in descending order of delay risk index to form an initial risk sequence, then the urgent tasks whose delay risk index is greater than a risk threshold are identified from the initial risk sequence, and the execution position of these urgent tasks is advanced in the second processing sequence to finally generate a target processing sequence. This mechanism ensures that tasks with high delay risk can be processed more timely, effectively reduces the possibility of task timeout, and reduces the business impact that may be caused by task processing delay. While ensuring the overall processing efficiency, the response capability of the system to the business timeliness requirement is improved, and the risk priority principle of task processing is realized.

[0017] Optionally, after the plurality of business tasks are processed in the target processing sequence by the preset global task listener, the method further comprises: monitoring the actual execution progress of each business task in real time; and when it is detected that the actual execution progress of any business task falls behind the preset progress, adjusting the target processing sequence to generate a final processing sequence.

[0018] By adopting the technical scheme, the real-time monitoring and dynamic adjustment mechanism is introduced, and the continuous optimization of the task processing process is realized. When the global task listener executes the target processing sequence, the actual execution progress of each business task is monitored in real time. When it is found that the execution progress of any task falls behind the preset progress, the system will timely adjust the target processing sequence to generate the final processing sequence. This closed-loop feedback mechanism enables the task scheduling system to have self-adaptive ability, and can respond and correct in real time to abnormal situations occurring in the execution process, prevent the task execution from deviating from the expectation and causing chain delay, ensure that the overall task processing continuously maintains the optimal state, and further improve the reliability and adaptability of the system in processing tasks. Especially when facing complex and changeable business environment, it can more flexibly cope with various execution abnormalities.

[0019] In a second aspect, the present application provides a data processing system, comprising: a receiving module, a first obtaining module, a second obtaining module, a calculating module and an adjusting module; wherein,

[0020] The receiving module is configured to receive a plurality of task creation requests sent by a business system, and create a plurality of business tasks. The first obtaining module is configured to obtain a processing priority of each business task and an association index between each business task, and generate a first processing sequence of the plurality of business tasks in combination with the processing priority and the association index. The second obtaining module is configured to obtain a resource occupation amount of each business task, and adjust the first processing sequence according to the resource occupation amount to generate a second processing sequence. The calculating module is configured to calculate a delay risk index of each business task, and adjust the second processing sequence according to the delay risk index to generate a target processing sequence. The delay risk index is used to represent the degree of influence of the processing delay of each business task on the business. The adjusting module is configured to process the plurality of business tasks according to the target processing sequence through a preset global task listener.

[0021] In a third aspect, the present application provides an electronic device, adopting the following technical scheme: comprising a processor, a memory, a user interface and a network interface, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to execute the computer program of any of the above data processing methods.

[0022] In a fourth aspect, the present application provides a computer readable storage medium, adopting the following technical scheme: storing a computer program capable of being loaded and executed by a processor to execute any of the above data processing methods.

[0023] In summary, the present application has at least one of the following beneficial technical effects:

[0024] By comprehensively considering the processing priority of business tasks, the correlation index between tasks, the resource occupation amount, and the delay risk index, a multi-dimensional task scheduling mechanism is constructed, and intelligent optimization of the processing order of business tasks is realized. The scheme first establishes an initial order based on business importance, then ensures the cooperative processing of related tasks through correlation analysis, adjusts to avoid system resource bottlenecks in combination with resource occupation amount, and finally further optimizes the processing order according to the delay risk index, comprehensively balancing multiple factors such as business priority, task correlation, system resource utilization efficiency, and business impact degree. The preset global task listener executes the optimized target processing order, significantly improving the scheduling efficiency in complex business scenarios, thereby improving the overall performance of the system. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flow diagram of a data processing method provided by an embodiment of the present application;

[0026] Figure 2 is a structural diagram of a data processing system provided by an embodiment of the present application;

[0027] Figure 3 is a structural diagram of an electronic device provided by an embodiment of the present application.

[0028] The following items are explained: 1000, electronic device; 1001, processor; 1002, communication bus; 1003, user interface; 1004, network interface; 1005, memory. DETAILED DESCRIPTION

[0029] In order to enable personnel in the technical field to better understand the technical solutions in the present specification, the technical solutions in the present specification will be clearly and completely described below in conjunction with the drawings in the embodiments of the present specification. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.

[0030] In the description of the embodiments of the present application, the words "exemplary", "for example", or "for instance" are used to mean serving as an example, instance, or illustration. Any embodiment or design solution described as "exemplary", "for example", or "for instance" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. Rather, the words "exemplary", "for example", or "for instance" are used in the sense of presenting a specific way of implementing the relevant concept.

[0031] The application scenario of the present application includes but is not limited to that a certain telecom operator uses a Flowable workflow engine to manage a customer order processing system, and faces thousands of business processing tasks of different types, priorities and resource requirements every day, such as number porting, package changing, complaint processing, etc. The system originally uses a simple first-in-first-out strategy to process tasks, resulting in problems such as delay of high-value services, uneven allocation of system resources, and fragmented processing of related services. The present application deploys a global task listener, and the system can automatically analyze the processing priority, correlation index, resource occupation and delay risk index of each order, dynamically generate the optimal processing order, prioritize VIP customer orders, execute related services coherently, stagger high-resource-consumption tasks, and timely advance near-time-out tasks, thereby ensuring balanced use of system resources, improving task processing efficiency and customer satisfaction, and especially during peak periods, the system no longer collapses due to resource contention, achieving intelligent, automated and fine-grained management of business processing.

[0032] Figure 1 is a flow diagram of a data processing method provided by an embodiment of the present application. As shown in Figure 1 , the method comprises S101-S105:

[0033] S101, receiving a plurality of task creation requests sent by a business system, and creating a plurality of business tasks.

[0034] In the present embodiment, the business system can be an OA system, a customer relationship management system, a smart city management platform, etc. in an enterprise, and various business applications. These systems will generate a large number of business tasks that need to be processed in daily operation, such as customer order processing, document approval, complaint processing, city event work order, etc. In the traditional workflow, these tasks are usually processed according to a simple time sequence or fixed priority, lacking intelligent scheduling mechanism, resulting in unbalanced system resource utilization and possible delay of high-value services.

[0035] To solve this problem, the present application receives task creation requests sent by these business systems through a system interface, each request containing task type, business data, priority identifier, expected completion time, etc. The system then converts these requests into standardized business task objects, each business task object containing a unique identifier, business type, creation time, expected completion time, task status, etc. attribute field, and establishes the association between the task and the original business data, ensuring that the business information can be accurately traced and updated in the subsequent processing process.

[0036] In actual implementation, the system can capture these task creation events through a global task listener based on the Flowable workflow engine, which listens to the TaskCreatedEvent event type in the Flowable engine by configuration, to realize centralized listening and processing of all user task creation processes.

[0037] Flowable is an open-source business process management platform that provides a complete workflow engine and business process execution environment for designing, executing, and monitoring business processes. It is based on the BPMN 2.0 (Business Process Model and Notation) standard, supports process definition, task assignment, event listening, and other core functions, and is widely used in the field of enterprise workflow automation.

[0038] TaskCreatedEvent is an event type in the Flowable workflow engine, which belongs to the Flowable event mechanism. When a new user task is created in the workflow, the Flowable engine will automatically trigger this event. The system can capture and process these events by configuring event listeners, to realize centralized monitoring and management of the task creation process.

[0039] S102, obtain the processing priority of each business task and the correlation index between each business task, and generate a first processing order of multiple business tasks by combining the processing priority and the correlation index.

[0040] In specific implementation, the system first obtains the preset priority of each business task from the task attribute or business rule library, such as the basic priority value set according to the task type (urgent task, regular task, batch task, etc.) or the business source (VIP customer, ordinary customer, etc.); then the system retrieves the task processing rule library, and adjusts the preset priority according to the current business scenario, system load, and other dynamic factors, to obtain the final processing priority of each business task. The processing priority can be represented as a value of 1-10, and the larger the value, the higher the task processing urgency. At the same time, the system analyzes the correlation between each business task and calculates the correlation index between tasks.

[0041] The association index refers to a value representing the degree of association between two business tasks, which can be determined in various ways: first, based on explicit definition of business rules, such as multiple requests from the same customer having high association; second, through data analysis to mine implicit associations, such as city event work orders with close geographical proximity may have association; third, based on learned association patterns from historical processing data. The association index can be represented as a value between 0 and 1, with a larger value indicating a higher degree of association between tasks. After obtaining the two sets of parameters, the system first sorts the tasks according to the processing priority to determine the initial processing order; then identifies task pairs with an association index exceeding a preset threshold (such as 0.7), and adjusts the positions of these highly associated tasks in the initial order to make them as adjacent as possible, thereby forming the final first processing order.

[0042] Based on the above embodiments, as an optional implementation, in S102, obtaining the processing priority of each business task and the association index between each business task specifically includes S21-S22:

[0043] S21, obtaining the preset priority of each business task and the processing rule of each business task, adjusting the preset priority according to the processing rule to obtain the processing priority of each business task.

[0044] The system obtains the preset priority of each business task and the processing rule of each business task, and adjusts the preset priority according to the processing rule to obtain the processing priority of each business task. The preset priority is usually a basic priority value set by the business system when creating the task based on basic information such as business type, customer level, and transaction amount. However, these static preset values often cannot fully reflect the true urgency of the task under the current business environment, and need to be dynamically adjusted in combination with the processing rule.

[0045] The processing rule refers to a combination of a series of conditions and actions pre-configured in the system, which is a set of strategies for adjusting the priority of tasks in a specific business scenario. The processing rule can include time decay rules, load balancing rules, business continuity rules, and other types. The system first obtains the preset priority of each business task from the task database or business system interface, then retrieves the set of processing rules applicable to the current business environment from the rule library, and applies these rules to adjust the preset priority. For example, for tasks that have approached the deadline, the system applies the time decay rule to increase their priority; for multiple requests from the same important customer, the system applies the customer aggregation rule to appropriately increase their priority; for system load peak periods, the load balancing rule may be applied to reduce the priority of some non-critical tasks. Through this dynamic adjustment, the system obtains a processing priority that better reflects the actual business value, providing more accurate basis data for subsequent task sorting.

[0046] S22, collect preset association levels between each business task, and determine association indexes between each business task according to the preset association levels, the association indexes being positively correlated with the association levels.

[0047] The system collects preset association levels between each business task, and determines association indexes between each business task according to the preset association levels, the association indexes being positively correlated with the association levels. The association between business tasks is an important consideration factor for intelligent task sequencing, and continuous processing of tasks with high association can greatly improve processing efficiency and business continuity.

[0048] The preset association level refers to a level division representing the closeness of association between business tasks, which can be explicitly defined by business rules or extracted from historical data. The system collects the preset association levels through various channels: first, it obtains the explicitly defined association relationship from the business rule library, such as different business requests of the same customer, multiple work items of the same project, etc.; second, it extracts the implicit association from the task metadata, such as work orders with close geographical location, tasks using the same resources, etc.; third, it analyzes the experience association from historical processing records, such as task types that are often processed continuously by the same processing personnel. The system then converts the preset association levels into standardized association indexes, which are numerical values between 0 and 1, used to quantitatively represent the degree of association between two tasks, with a larger value indicating a higher degree of association. The conversion process usually uses a mapping function, such as linear mapping or piecewise function, to ensure that the association index is positively correlated with the association level. In addition, the system also considers the transitivity and directionality of the association. For task groups with association transitivity, the system calculates the indirect association degree between tasks through graph algorithms to obtain a more comprehensive association index matrix.

[0049] To determine the association indexes between business tasks, the following fixed method can be used: first, divide the preset association levels into five standard levels: no association (level 1), weak association (level 2), moderate association (level 3), strong association (level 4), and close association (level 5). Then, apply a linear mapping function to convert the association level to the association index, with the specific calculation formula being: association index = (association level - 1) x 0.25, so that no association corresponds to an association index of 0, weak association corresponds to 0.25, moderate association corresponds to 0.5, strong association corresponds to 0.75, and close association corresponds to 1.0. For implicit association relationships, the system quantitatively evaluates them through four core dimensions (data sharing degree, business process continuity, resource usage similarity, and spatio-temporal correlation), with each dimension having a weight distribution of 0-0.25, and the cumulative score determining the final association index.

[0050] On the basis of the above embodiments, as an optional implementation, in S102, the first processing order of the plurality of business tasks is generated in combination with the processing priority and the association index, specifically including S23-S26:

[0051] S23, determining an initial processing order of the plurality of service tasks according to the processing priorities.

[0052] S24, determining pairs of service tasks with a correlation index greater than a correlation threshold.

[0053] The correlation threshold refers to a correlation index threshold value preset by the system. When the correlation index between two service tasks is greater than the threshold value, it indicates that the two tasks have significant correlation and should be arranged for processing in adjacent positions to improve processing efficiency. Based on the correlation index matrix calculated in step S22, the system compares the correlation index of each pair of service tasks with the preset correlation threshold (such as 0.75), identifies all pairs of tasks with a correlation index exceeding the threshold, and sorts them in descending order of correlation index to form a high-correlation task pair list. This step ensures that the system can identify closely related tasks that need to be processed continuously, providing a basis for subsequent order adjustment.

[0054] S25, in the initial processing order, adjusting the positions of the pairs of service tasks with a correlation index greater than the correlation threshold to be adjacent.

[0055] The system adjusts the positions of the pairs of service tasks with a correlation index greater than the correlation threshold to be adjacent in the initial processing order. For the identified high-correlation task pairs, the system needs to adjust their positions to be adjacent while maintaining the initial priority order as much as possible.

[0056] The specific adjustment strategy includes: first, processing the high-correlation task pairs in pairs in descending order of correlation index; for each pair of high-correlation tasks, comparing their positions in the initial processing order and adjusting the task with a later position to the position immediately after the task with an earlier position; if a conflict occurs during the adjustment process (i.e., multiple tasks need to be adjusted to the same position), the order is determined according to the size of the correlation index. To prevent tasks with low priority from being moved too early due to correlation, the system also sets a priority tolerance parameter to limit the maximum number of positions that a task can be moved forward during the adjustment process.

[0057] S26, generating a first processing order of the plurality of service tasks based on the adjusted order.

[0058] S103, obtaining the resource occupation of each service task, adjusting the first processing order according to the resource occupation, and generating a second processing order.

[0059] In the specific implementation process, the system first obtains the resource occupation information of each service task through the resource evaluation module. The resource occupation refers to the total amount of various resources consumed to complete a specific service task, which can be represented as a multi-dimensional resource vector, including but not limited to processor time, memory space, storage capacity, network bandwidth, special device usage time, and human resources, etc.

[0060] The method for obtaining the resource occupation amount comprises: 1) based on historical statistical data, average resource consumption is extracted by analyzing historical execution records of similar tasks; 2) based on task feature prediction, resource demand is estimated according to task type, data volume, complexity and other features; and 3) the resource demand declaration provided by the business system directly is obtained. After obtaining the resource occupation amount, the system calculates the resource use distribution of adjacent tasks in the first processing order, and identifies the region where the resource use presents a "peak", i.e. the section where multiple resource-intensive tasks are arranged in clusters. Subsequently, the system adjusts the processing order of these regions: the high resource consumption tasks whose resource occupation amount exceeds a preset threshold are identified, and under the premise of maintaining the relative order of task priorities and the relative positions of highly correlated tasks, these tasks are appropriately dispersed and inserted into tasks with lower resource occupation, forming a second processing order with more balanced resource use. In the adjustment process, the system uses a resource smoothing algorithm which comprehensively considers the peak values of various resources, task importance and correlation, and finds the task ordering scheme with the most balanced resource use through a dynamic programming method.

[0061] On the basis of the above embodiment, as an optional implementation, in S103, the first processing order is adjusted according to the resource occupation amount to generate the second processing order, which specifically comprises S31-S34:

[0062] S31, the total amount of system resources is obtained, and the ratio of the resource occupation amount of each business task to the total amount of system resources is calculated.

[0063] S32, high resource consumption tasks with a resource occupation rate greater than a resource threshold are determined.

[0064] S33, in the first processing order, the high resource consumption tasks are reordered so that tasks with a resource occupation rate less than the resource threshold are inserted between multiple high resource consumption tasks executed adjacently.

[0065] S34, the second processing order is generated based on the adjusted order.

[0066] The system reorders the high resource consumption tasks in the first processing order so that tasks with a resource occupation rate less than the resource threshold are inserted between multiple high resource consumption tasks executed adjacently. The system first analyzes the distribution of high resource consumption tasks in the first processing order and identifies all clusters of continuously arranged high resource consumption tasks. For each such cluster, the system calculates the peak resource occupation rate that may be caused if the tasks in the cluster are executed in the original order, and if the peak resource occupation rate exceeds the acceptable upper limit of the system (such as 80%), the tasks in the cluster need to be split and reordered.

[0067] The strategy of reordering is: under the premise of maintaining the overall framework of the first processing order, disperse the continuous high resource consumption tasks, and insert low resource occupancy tasks as "buffer" between them. The selection of inserted tasks considers multiple factors, including resource complementarity (i.e. selecting tasks that occupy different types of resources), processing priority (preferentially selecting low resource tasks with higher priority), and correlation index (trying to avoid disrupting high correlation tasks).

[0068] In S104, the delay risk index of each business task is calculated, and the second processing order is adjusted according to the delay risk index to generate a target processing order; the delay risk index is used to represent the degree of impact of the processing delay of each business task on the business.

[0069] In the actual business environment, the consequences of processing delays of different tasks are significantly different. Some tasks may cause serious business losses, compliance risks, and even chain reactions, while some tasks have high time tolerance. Although the second processing order formed based on priority, correlation index, and resource occupancy has considered the importance of tasks and system resource balance, it has not fully reflected the actual impact of task delay on the business, which may cause some time-sensitive tasks to be delayed due to scheduling, resulting in unnecessary business losses.

[0070] This step optimizes the processing order by introducing the key parameter of delay risk index, aiming to minimize the negative impact of task delay on the overall business and maximize business value. The delay risk index is a quantitative indicator representing the degree of negative impact of business task processing delay on related business, which is a comprehensive evaluation of the time sensitivity of the task. When calculating the delay risk index, the system considers multiple factors: first, the time sensitivity of the task, i.e. the rate of decrease in the value of the task over time, some tasks such as emergency handling and real-time transactions have high time sensitivity; second, the business impact range of the task, evaluating the size of the business range that may be affected by the delay of the task, the delay risk of tasks with wide impact is higher; third, the chain reaction risk, the delay of some tasks may trigger a series of subsequent problems, forming a chain effect; finally, it also includes factors such as compliance risk and customer satisfaction impact.

[0071] The system calculates the delay risk index by using a comprehensive evaluation algorithm, which can be expressed as a value between 0 and 100, and the higher the value, the greater the delay risk. After obtaining the delay risk index of each task, the system optimizes and adjusts the second processing sequence: first, identify high-risk tasks with a delay risk index exceeding a preset threshold (such as 85 points), and arrange these tasks in advance based on the aforementioned constraints; then apply time value decay calculation to the remaining tasks, calculate the expected loss of delayed processing according to the expected processing time and delay risk index of the task, and adjust the processing sequence accordingly to form the final target processing sequence.

[0072] Based on the above embodiments, as an optional implementation, in S104, calculating the delay risk index of each business task specifically includes S41-S42:

[0073] S41, obtain the preset business impact level of each business task, which is used to represent the impact of business task delay on the business process.

[0074] The system obtains the preset business impact level of each business task, which is used to represent the impact of business task delay on the business process. The business impact level is a grading index predefined by business experts or the system based on task characteristics and business rules, reflecting the negative impact that the delay of the task may have on the enterprise. The system usually divides the business impact level into multiple levels, such as critical (level 5), high importance (level 4), important (level 3), general (level 2), and low impact (level 1). The system obtains the business impact level through multiple channels: one is to extract the default impact level based on task type from the business rule library; two is to obtain the impact rating of specific tasks from business department or customer feedback; three is to automatically derive the impact level based on the correlation between task delay and business indicators from historical data analysis. The introduction of business impact level enables the system to distinguish the risk differences of different tasks, providing a key input for the calculation of delay risk index.

[0075] S42, obtain the remaining processing time limit of each business task, and calculate the delay risk index of each business task according to the business impact level and the remaining processing time limit, wherein the business impact level is positively correlated with the delay risk index, and the remaining processing time limit is inversely correlated with the delay risk index.

[0076] The system obtains the remaining processing time limit of each business task, and calculates the delay risk index of each business task according to the business impact level and the remaining processing time limit. The remaining processing time limit refers to the time interval from the current time point to the deadline that the task must be completed, usually in hours or minutes. The system first extracts the predetermined deadline from the task metadata, and calculates the remaining processing time limit in combination with the current system time. For tasks without a specific deadline, the system determines the standard processing time limit based on the service level agreement (SLA) and the task type. The delay risk index is a comprehensive index that quantifies the business risk that may be caused by the delay of the task processing. The numerical range is usually 0-100, and the higher the value, the greater the delay risk.

[0077] The system uses the following calculation formula: delay risk index = business impact weight × (1 - remaining time limit adjustment factor), where the business impact weight is the weight value of the business impact level mapped to the 0-100 interval, such as 100 for the critical level, 80 for the highly important level, and so on. The remaining time limit adjustment factor is a value between 0 and 1, which is calculated by applying an inverse proportional function to the remaining processing time limit, for example, the calculation method of remaining time limit adjustment factor = min(1, remaining processing time limit / standard processing time) can be used. This calculation method ensures that the business impact level and the delay risk index are positively correlated, i.e. the higher the impact level, the greater the delay risk index; at the same time, the remaining processing time limit and the delay risk index are inversely related, i.e. the shorter the remaining time, the greater the delay risk index. For special types of tasks, the system also considers additional factors such as nighttime batch processing windows, business peak periods, system maintenance periods, and other time factors that affect the delay risk, and adjusts them by introducing a correction coefficient in the basic formula.

[0078] On the basis of the above embodiment, as an optional implementation, in S104, the second processing order is adjusted according to the delay risk index to generate a target processing order, which specifically includes S43-S45:

[0079] S43, arranging the business tasks in the second processing order in descending order according to the delay risk index to generate an initial risk order.

[0080] S44, identifying an urgent task in the initial risk order whose delay risk index is greater than a risk threshold.

[0081] S45, advancing the execution position of the urgent task in the second processing order to generate a target processing order.

[0082] The system advances the execution position of the urgent task in the second processing order to generate a target processing order. The system needs to advance the urgent task identified in S44 as much as possible while maintaining the overall structure of the second processing order to reduce its delay risk.

[0083] The specific adjustment strategy includes: first, processing each emergency task from high to low according to the delay risk index; for each emergency task, the system calculates the minimum position number that needs to be advanced according to its current position in the second processing order and the estimated time required for completion, so that it can be completed before the deadline; the system adopts the principle of minimum intervention when adjusting, that is, on the premise of meeting the risk control target, it tries to reduce the damage to the original order as much as possible, especially avoiding damaging the already optimized associated task pairs and resource balanced distribution; for the case that multiple emergency tasks need to be adjusted to similar positions, the system determines the final position according to the size of the delay risk index, and the task with higher risk index is ranked earlier. The system also sets an upper limit parameter for position advance to prevent low-priority but high-risk tasks from being advanced too much, which may cause damage to the overall business value.

[0084] After such fine position adjustment, the system generates the final target processing order, which is further optimized on the basis of the second processing order, maintaining the basic framework of priority, relevance and resource balance, and effectively reducing the possibility of delay of high-risk tasks.

[0085] S105, processing the plurality of business tasks according to the target processing order through a preset global task listener.

[0086] The system processes the plurality of business tasks according to the target processing order through a preset global task listener. In the traditional task processing mechanism, task processing is often scattered in various business systems and executed independently, lacking unified monitoring and scheduling capabilities, and unable to effectively execute the optimized processing order generated in the previous steps, resulting in unreasonable resource allocation, low processing efficiency and other problems.

[0087] By introducing a global task listener as a unified coordination mechanism for task processing, the optimized target processing order is effectively applied to the actual business processing process, ensuring that tasks can be executed in the best order, thereby maximizing the overall processing efficiency and business value of the system. The global task listener refers to a system-level component that can monitor and coordinate task processing flows across multiple business systems, with functions such as task state monitoring, resource scheduling, execution order control, etc.

[0088] In a specific implementation, the global task listener first integrates with the workflow engine, and monitors the entire lifecycle of a task by configuring to listen to task events in the workflow engine. After receiving the target processing sequence generated in the foregoing step, the listener converts it into a task processing instruction sequence and stores it in the system's scheduling queue, and then coordinates the processing of each task according to the sequence of the queue. The coordination methods include: 1) controlling the task allocation mechanism of the workflow engine to ensure that the system allocates tasks to the execution personnel or automatic processing components according to the target processing sequence; 2) reasonably scheduling the task execution resources to ensure that high-priority tasks can obtain sufficient processing resources; 3) monitoring the task processing state in real time, warning and intervening when a task is delayed, and adjusting the processing sequence of subsequent tasks as necessary to adapt to real-time changes.

[0089] The global task listener also maintains the dependency relationship between tasks and the resource occupation situation. When a resource-intensive task is completed, the listener can immediately release the related resources and allocate them to subsequent tasks, ensuring the continuity and efficiency of resource utilization. In addition, the listener collects and records the entire process data of task processing, including task start time, completion time, resource occupation situation, execution result, etc., to provide data support for subsequent task processing optimization.

[0090] After the plurality of business tasks are processed according to the target processing sequence by the preset global task listener, the method further includes:

[0091] monitoring the actual execution progress of each business task in real time; when it is detected that the actual execution progress of any business task falls behind the preset progress, adjusting the target processing sequence to generate a final processing sequence.

[0092] The actual execution progress of each business task is monitored in real time by the global task listener. The global task listener is a special component preset in the system, responsible for collecting and analyzing state data in the task execution process. The listener obtains execution progress information in multiple ways: 1) from the task execution engine to obtain task state update events; 2) by periodically polling the state data of the task execution environment; 3) receiving progress information actively reported by the task processing component. The system sets an expected execution path and time node for each task, forming a preset progress baseline. The preset progress refers to the ideal execution time plan formulated by the system for each task based on historical execution data and task characteristic analysis, including the predicted start time, key stage completion time, and predicted end time. The global task listener compares the real-time collected task execution data with the preset progress, calculates the progress deviation, and evaluates the potential impact of the deviation on the overall processing flow.

[0093] When the system detects that the actual execution progress of any business task lags behind the preset schedule, it will trigger a dynamic adjustment of the processing order. The system first performs an impact analysis on the lagging task, assessing the degree of impact of its delay on subsequent tasks and overall business objectives. The assessment considers several factors: first, the criticality of the delayed task, determining whether it is located on the critical path of the business; second, the severity of the delay, calculating the deviation between the actual progress and the preset schedule; third, the scope of the associated impact, analyzing how many subsequent tasks depend on the completion of this task; and fourth, the remaining processing time, assessing whether the task can be completed as originally planned within the remaining time. Based on these analyses, the system calculates an adjustment urgency index for each delayed task, guiding subsequent order adjustments.

[0094] The system adopts corresponding processing strategies based on the urgency of adjustments. For tasks with high urgency, the system may take the following measures: first, resource optimization, allocating more system resources to the task to accelerate processing; second, order adjustment, moving the task to the front of the current execution queue; third, parallel processing, splitting the task into parallelizable subtasks if conditions permit; and fourth, task replacement, temporarily suspending currently executing non-critical tasks and prioritizing high-urgency tasks. For tasks with medium urgency, the system may adopt moderate adjustment strategies, such as fine-tuning its position in the execution queue or reserving necessary system resources for it in advance. For tasks with low urgency, the system may temporarily maintain the original plan but increase the frequency of monitoring its execution status to prepare for possible future adjustments.

[0095] Through these dynamic adjustment strategies, the system generates the final processing sequence. This final processing sequence refers to the actual sequence of business task processing adopted by the system after real-time monitoring and dynamic adjustment. It is a targeted optimization of the target processing sequence to adapt to changes in the actual situation during execution. The system applies the final processing sequence to the task scheduling engine to guide the execution allocation and resource scheduling of subsequent tasks. Simultaneously, the system continuously monitors the adjusted execution effect and feeds relevant data back to the prediction model and rule base to optimize the future processing sequence generation process.

[0096] Based on the above method, this application also discloses a data processing system, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application. The system includes: a receiving module, a first acquisition module, a second acquisition module, a calculation module, and an adjustment module; wherein,

[0097] The receiving module is configured to receive a plurality of task creation requests sent by a service system and create a plurality of service tasks; the first obtaining module is configured to obtain a processing priority of each service task and an association index between the service tasks, and generate a first processing sequence of the service tasks in combination with the processing priority and the association index; the second obtaining module is configured to obtain a resource occupation amount of each service task, adjust the first processing sequence according to the resource occupation amount, and generate a second processing sequence; the calculating module is configured to calculate a delay risk index of each service task, adjust the second processing sequence according to the delay risk index, and generate a target processing sequence; the delay risk index is used to represent a degree of influence of processing delay of each service task on a service; and the adjusting module is configured to process the plurality of service tasks in the target processing sequence through a preset global task listener.

[0098] It should be noted that: the system provided in the above embodiments, when realizing its functions, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0099] Please refer to Figure 3 A structural schematic diagram of an electronic device is provided for the embodiments of the present application. As shown in the figure Figure 3 The electronic device 1000 can include at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0100] The communication bus 1002 is used to realize the connection and communication between the components.

[0101] The user interface 1003 can include a display screen (Display) and a camera (Camera), and the optional user interface 1003 can also include a standard wired interface and a wireless interface.

[0102] The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0103] The processor 1001 can include one or more processing cores. The processor 1001 connects various parts within the server through various interfaces and lines, and performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Alternatively, the processor 1001 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1001 can integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 1001, but can be realized by a separate chip.

[0104] The memory 1005 can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 1005 can also be at least one storage device located away from the aforementioned processor 1001. As shown, the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an application program of a data processing method. Figure 3

[0105] In Figure 3 ​In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor 1001 can be used to call an application program stored in the memory 1005 and storing a data processing method, which, when executed by one or more processors, causes the electronic device to perform the method described in one or more of the above embodiments.

[0106] An electronic device readable storage medium stores instructions that, when executed by one or more processors, cause an electronic device to perform the method described in one or more of the above embodiments.

[0107] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0108] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0109] In several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some services, interfaces, devices or units, and can be electrical or other forms.

[0110] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0111] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0112] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned memory includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0113] The above-described are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional techniques in the art that are not described in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A data processing method, characterized by, The method comprises: receiving a plurality of task creation requests sent by a business system, creating a plurality of business tasks; obtaining the processing priority of each business task and the association index between each business task, including: obtaining the preset priority of each business task and the processing rule of each business task, adjusting the preset priority according to the processing rule to obtain the processing priority of each business task; collecting the preset association level between each business task, determining the association index between each business task according to the preset association level, and the association index is positively correlated with the association level; combining the processing priority and the association index to generate a first processing order of a plurality of business tasks, including: determining the initial processing order of a plurality of business tasks according to the processing priority; determining the business task pair whose association index is greater than the association threshold; in the initial processing order, the positions of the business task pair whose association index is greater than the association threshold are adjusted to be adjacent; based on the adjusted order, a first processing order of a plurality of business tasks is generated; the association index refers to a numerical value representing the degree of association between two business tasks, which can be determined in various ways, including: explicit definition based on business rules, implicit association mining through data analysis, and association mode learned based on historical processing data, the association index is represented as a numerical value between 01, and the larger the numerical value is, the higher the degree of association between tasks is; obtaining the resource occupation of each business task, and adjusting the first processing order according to the resource occupation to generate a second processing order; calculating the delay risk index of each business task, including: obtaining the preset business impact level of each business task, the business impact level is used to represent the influence degree of business task delay on business process; obtaining the remaining processing time limit of each business task, calculating the delay risk index of each business task according to the business impact level and the remaining processing time limit, wherein the business impact level is positively correlated with the delay risk index, and the remaining processing time limit is inversely correlated with the delay risk index; adjusting the second processing order according to the delay risk index to generate a target processing order, including: arranging each business task in the second processing order in descending order according to the delay risk index to generate an initial risk order; identifying the urgent task whose delay risk index is greater than the risk threshold in the initial risk order; advancing the execution position of the urgent task in the second processing order to generate a target processing order; the delay risk index is used to represent the degree of influence of the processing delay of each business task on the business; processing a plurality of business tasks according to the target processing order through a preset global task listener.

2. The data processing method according to claim 1, characterized in that, The adjusting the first processing sequence according to the resource occupation quantity comprises: obtaining a total amount of system resources, calculating a ratio of the resource occupation quantity of each service task to the total amount of system resources; determining a high resource consumption task with a resource occupation rate greater than a resource threshold; reordering the high resource consumption task in the first processing sequence, so as to insert a task with a resource occupation rate less than the resource threshold between multiple high resource consumption tasks that are executed adjacently; and generating a second processing sequence based on the adjusted sequence.

3. The data processing method of claim 1, wherein, After the multiple service tasks are processed according to the target processing sequence by the preset global task listener, the method further comprises: monitoring actual execution progress of each service task in real time; and when it is detected that actual execution progress of any service task lags behind a preset progress, adjusting the target processing sequence to generate a final processing sequence.

4. A data processing system, characterized by The system comprises a receiving module, a first obtaining module, a second obtaining module, a calculating module and an adjusting module; the receiving module is configured to receive a plurality of task creation requests sent by a business system, and create a plurality of business tasks; the first obtaining module is configured to obtain a processing priority of each of the business tasks and an association index between the business tasks, including: obtaining a preset priority of each of the business tasks and a processing rule of each of the business tasks, adjusting the preset priority according to the processing rule to obtain the processing priority of each of the business tasks; collecting a preset association level between the business tasks, and determining the association index between the business tasks according to the preset association level, the association index being positively correlated with the association level; combining the processing priority and the association index to generate a first processing sequence of the business tasks, including: determining an initial processing sequence of the business tasks according to the processing priority; determining a pair of business tasks whose association index is greater than an association threshold; adjusting the positions of the pair of business tasks whose association index is greater than the association threshold to be adjacent in the initial processing sequence; and generating the first processing sequence of the business tasks based on the adjusted sequence; the association index is a value representing the degree of association between two business tasks, which can be determined in various ways, including: explicitly defined based on business rules, implicitly associated by data analysis, and associated mode learned based on historical processing data, the association index being represented as a value between 0 and 1, and the greater the value, the higher the degree of association between tasks; the second obtaining module is configured to obtain a resource occupation amount of each of the business tasks, and adjust the first processing sequence according to the resource occupation amount to generate a second processing sequence; the calculating module is configured to calculate a delay risk index of each of the business tasks, including: obtaining a preset business impact level of each of the business tasks, the business impact level being used to represent the degree of influence of the delay of the business task on the business process; obtaining a remaining processing time limit of each of the business tasks, and calculating a delay risk index of each of the business tasks according to the business impact level and the remaining processing time limit, wherein the business impact level is positively correlated with the delay risk index, and the remaining processing time limit is inversely correlated with the delay risk index; adjusting the second processing sequence according to the delay risk index to generate a target processing sequence, including: arranging each of the business tasks in the second processing sequence in descending order of the delay risk index to generate an initial risk sequence; identifying an urgent task whose delay risk index is greater than a risk threshold in the initial risk sequence; and advancing the execution position of the urgent task in the second processing sequence to generate the target processing sequence; the delay risk index is used to represent the degree of influence of the processing delay of each of the business tasks on the business; and the adjusting module is configured to process the plurality of business tasks in the target processing sequence through a preset global task listener.

5. An electronic device, comprising: An electronic device comprising a processor, a memory for storing instructions, a user interface and a network interface for communicating with other devices, the processor being configured to execute the instructions stored in the memory to cause the electronic device to perform the method of any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, A computer program stored in a memory and loadable into the working memory of a digital computer, comprising software code portions arranged to make the computer execute the method of any one of claims 1-3 when said product is run on the computer.

Images