A log analysis task scheduling system based on neural network

Abstract

The present invention relates to the field of task scheduling technology, and specifically to a log analysis task scheduling system based on a neural network. In the present invention, by extracting task resource requirements and task execution duration from a database and mapping them to a task resource dependency graph, it is possible to achieve accurate matching of tasks and resources, optimize task scheduling efficiency, and establish dependencies between tasks, so that the optimization of task execution sequence is more flexible and real-time. By accurately analyzing the priority and time window of tasks through a long short-term memory neural network, it is possible to efficiently estimate task execution time, optimize the execution sequence of tasks, and reasonably arrange the order of concurrent execution of tasks under limited resources. By applying graph neural network technology in resource allocation, the generation of resource allocation plans is made more intelligent, and it is possible to judge in real time whether the current system resources are sufficient to support task execution, and adjust the order of task execution in real time, so as to avoid task blocking or delay caused by resource bottlenecks.

Description

Technical Field

[0001] The present invention relates to the technical field of task scheduling, and in particular to a log analysis task scheduling system based on neural network. Background Art

[0002] The goal of the task scheduling technology field is to optimize resource utilization, reduce task execution time, improve overall system performance, and effectively distribute multiple tasks and workloads to different processing units so that tasks can be executed efficiently under limited resources.

[0003] A neural network-based log analysis task scheduling system aims to automatically analyze log data through deep learning technology, identify key patterns and potential problems, and automatically schedule related tasks based on the analysis results to achieve more efficient log management and analysis. It aims to improve the processing efficiency of log analysis tasks and optimize resource allocation through intelligent scheduling, thereby improving the stability and reliability of system operation.

[0004] Existing technologies lack in-depth analysis of task dependencies. The order of task execution is often fixed and cannot be flexibly adjusted according to the actual dependencies and priorities between tasks. This leads to task delays and resource conflicts under high load conditions. It is unable to effectively handle complex dependencies between tasks and cannot provide real-time feedback on the actual needs and resource status during task execution, resulting in inefficient resource allocation and long task execution time, affecting the overall performance and stability of the system. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a log analysis task scheduling system based on neural network.

[0006] In order to achieve the above objectives, the present invention adopts the following technical solutions: A log analysis task scheduling system based on neural network includes:

[0007] Graph construction module: This module extracts task metadata from the database, obtains the execution duration and required resources of each task, maps tasks and resource nodes, establishes dependencies between tasks as edges, determines whether to add new edges based on the dependency strength, and adds task priority and resource quota information as additional attributes of the nodes to construct a task resource dependency graph.

[0008] Dependency parsing module: Based on the task resource dependency graph, obtain the mutual dependency relationship of all task nodes, calculate the dependency matrix between nodes, determine the dependency relationship between each pair of nodes, sort the nodes by dependency strength, adjust the node priority when the dependency strength exceeds the threshold, and determine whether the task can be executed, and generate a task execution dependency list;

[0009] Sequence prediction module: Based on the task execution dependency list, the long short-term memory neural network is used to analyze the priority and time window sequence of the task, estimate the execution time of the task, determine the appropriate task execution order, sort the task execution order according to the priority, perform time sequence scheduling based on the execution cycle and priority, and combine the delay determination and resource requirements between tasks to generate a task execution sequence table;

[0010] Resource allocation module: Based on the task execution sequence table, a graph neural network is used to extract the resource requirements of each task in turn, compare the current resource status, determine whether there are sufficient resources to meet the concurrent execution of tasks, and adjust resource allocation according to task priority. If the current resources are insufficient, some tasks are temporarily suspended according to their priority. Resources are divided into multiple priorities and a resource allocation optimization plan is generated;

[0011] Execution control module: Based on the resource allocation optimization plan, it determines whether each resource meets the execution requirements of the task. If the resource is available, it starts the task execution and registers the execution status. If the resource is insufficient, it pauses the task and reschedules it. It continuously monitors the status changes of each task, updates the task execution mark, and generates a task execution status report.

[0012] As a further solution of the present invention, the graph construction module includes:

[0013] Task metadata extraction submodule: extracts task metadata from the database, queries the database at the same time, obtains relevant information of all tasks, stores it in the data table, and generates a task metadata table;

[0014] Node mapping submodule: Based on the task metadata table, tasks and required resources are mapped into nodes in the graph, each task corresponds to a task node, each resource corresponds to a resource node, and a task-resource node mapping table is generated by connecting the task nodes with the resource nodes;

[0015] Dependency construction submodule: Based on the task resource node mapping table, the dependency relationship between tasks is obtained, and directed edges between tasks are established through the dependency identification between tasks. It is determined whether new edges need to be added according to the dependency strength. If the dependency strength is higher than the predetermined threshold, the edge connection is added to generate a task resource dependency graph.

[0016] As a further solution of the present invention, the dependency parsing module includes:

[0017] Relationship extraction submodule: Based on the task resource dependency graph, perform node traversal operations, read the start and end node identifiers of each edge in the graph and compare them with the task numbers, obtain the corresponding relationship between task nodes, extract the edge connection weight as the dependency strength value, construct the dependency weight matrix, and generate the node dependency weight set;

[0018] Prioritization submodule: Based on the node dependency weight set, a numerical comparison operation of dependency strength is performed. By summing the weight values ​​in the outgoing edge set of each node and sorting them by size, the strongly dependent nodes are extracted and corresponding priority labels are established. The priority arrangement in task scheduling is adjusted to generate a task priority mapping table.

[0019] Execution verification submodule: Based on the task priority mapping table, determine the executable status of each task node, compare the hierarchical position of the task in the topology structure and verify whether all predecessor nodes have unfinished dependencies, mark the executable and pending states, summarize the sorting nodes and status information, and generate a task execution dependency list.

[0020] As a further solution of the present invention, the sequence prediction module includes:

[0021] Temporal structure analysis submodule: Based on the task execution dependency list, a long short-term memory neural network is used to extract the priority, dependency node number, and time window boundary corresponding to each task node, perform time interval comparison operations, screen overlapping interval tasks, calculate priority differences, establish the time interaction structure of each task, and generate a time interaction matrix set;

[0022] Execution duration evaluation submodule: Based on the time interaction matrix set, the difference between the start time and the end time of each task is solved, and the actual duration of the task is determined in combination with the task resource usage density value, and the task execution duration is updated and redistributed in intervals to generate a task interval duration table;

[0023] Scheduling sequence generation submodule: Based on the task interval duration table, the priority, time span and required resources of all tasks are jointly sorted, and a scheduling starting point identification table is constructed through time continuity and resource distribution sparsity to determine the order of task execution, establish an executable process chain, and generate a task execution sequence table.

[0024] As a further solution of the present invention, the long short-term memory neural network is according to the formula:

[0025]

[0026] in: Indicates a task and tasks The priority gap between Indicates a task The start time, Indicates a task The end time, Indicates a task The start time, Indicates a task The end time, Indicates a task Priority, Indicates a task Priority, Indicates the weight coefficient of the priority difference calculation for the task time overlap. The weight coefficient that represents the impact of task priority differences on the calculation results, Indicates a task With the task The weight coefficient of task dependency on the calculation result, Indicates the weight coefficient of task start time difference to priority difference calculation, Indicates a task and tasks The dependency between tasks.

[0027] As a further solution of the present invention, the resource allocation module includes:

[0028] Resource requirement extraction submodule: Based on the task execution sequence table, a graph neural network is used to extract task indexes and locate task numbers. By reading the resource type identifier and resource quantity value in the corresponding field of the task, a number and value combination is constructed to match the relationship between the task and the required resources. The matching results are then registered in a unified record table to generate a task resource requirement list.

[0029] Allocation condition judgment submodule: Based on the task resource requirement list, the system resource status snapshot is read and a resource remaining quantity mapping table is established. The numerical difference between the task required resources and the remaining resources is calculated in sequence. The current resources are marked as meeting the task execution conditions. A queue mark is added to the insufficient resource items to generate a resource allocation feasibility mark set.

[0030] Resource priority division submodule: Based on the resource allocation feasibility tag set, extract all schedulable task numbers and corresponding priority values, divide the priority intervals and build a resource level queue, map the remaining system resources to each task group in proportion and register the mapping relationship between tasks and resources, and generate a resource allocation optimization plan.

[0031] As a further solution of the present invention, the graph neural network is according to the formula:

[0032]

[0033] in: Indicates a task The total resource requirements calculated by the graph neural network, Resource Type The unit resource consumption value, For the task Resource Type The quantity of demand, is the number of all resource types, Resource Type Task The weight coefficient of resource demand, For the task and resource types The dependence between is the adjustment weight coefficient of time resource consumption to total resource demand, To calculate the adjustment weight coefficient of resource consumption to total resource demand, For the task Time resource requirements, For the task computing resource requirements.

[0034] As a further solution of the present invention, the execution control module includes:

[0035] Task trigger judgment submodule: Based on the resource allocation optimization plan, it performs a joint query on the task and resource locking status, performs an availability judgment operation on the status field of the task-bound resource, marks the task in the non-occupied resource state as pending, creates a numbered list, and generates a task activation identification set;

[0036] Status record execution submodule: based on the task activation identification set, traverses all task numbers to be executed, obtains the execution start time and resource usage status, records the task execution time period and writes it into the task status field, sets the resource to be occupied, constructs a mapping relationship between the number and time, and generates a task execution status record table;

[0037] Execution result update submodule: Based on the task execution status record table, poll to obtain the current execution status value of the task, determine whether the task completion mark and the upper limit of the execution time meet the termination conditions, release the occupied resources and update the execution status code and task completion time, complete the status write operation, and generate a task execution status report.

[0038] Compared with the prior art, the advantages and positive effects of the present invention are:

[0039] 1. This invention extracts task resource requirements and task execution durations from a database and maps them to a task resource dependency graph. This allows for precise matching of tasks and resources, optimizes task scheduling efficiency, establishes inter-task dependencies, and dynamically adjusts task priorities based on dependency strength, making task execution order optimization more flexible and real-time.

[0040] 2. This invention uses a long short-term memory neural network to accurately analyze task priorities and time windows, effectively estimating task execution times, optimizing task execution order, and rationally arranging the order of concurrent task execution under limited resources.

[0041] 3. In this invention, by applying graph neural network technology to resource allocation, the generation of resource allocation plans is made more intelligent. This allows real-time determination of whether current system resources are sufficient to support task execution, and real-time adjustment of the order of task execution to avoid task blockage or delays caused by resource bottlenecks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a system flow chart of the present invention;

[0043] Figure 2 Schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0046] See also Figure 1 The present invention provides a technical solution: a log analysis task scheduling system based on a neural network includes:

[0047] Graph construction module: This module extracts task metadata from the database, obtains the execution duration and required resources of each task, maps tasks and resource nodes, establishes dependencies between tasks as edges, determines whether to add new edges based on the dependency strength, and adds task priority and resource quota information as additional attributes of the nodes to construct a task resource dependency graph.

[0048] Dependency parsing module: Based on the task resource dependency graph, it obtains the interdependencies of all task nodes, calculates the dependency matrix between nodes, determines the dependency between each pair of nodes, sorts nodes by dependency strength, adjusts node priority when the dependency strength exceeds a threshold, determines whether the task can be executed, and generates a task execution dependency list;

[0049] Sequence prediction module: Based on the task execution dependency list, it uses a long short-term memory neural network to analyze the task priority and time window sequence, estimate the task execution time, determine the appropriate task execution sequence, sort the task execution sequence according to priority, perform time sequence scheduling based on execution cycle and priority, and generate a task execution sequence table based on delay determination and resource requirements between tasks;

[0050] Resource allocation module: Based on the task execution sequence table, a graph neural network is used to extract the resource requirements of each task in turn, compare the current resource status, determine whether there are sufficient resources to meet the concurrent execution of tasks, and adjust resource allocation according to task priority. If the current resources are insufficient, some tasks are temporarily suspended according to their priority. Resources are divided into multiple priorities to generate a resource allocation optimization plan;

[0051] Execution control module: Based on the resource allocation optimization plan, it determines whether each resource meets the execution requirements of the task. If the resources are available, the task execution is started and the execution status is registered. If the resources are insufficient, the task is suspended and rescheduled. The status changes of each task are continuously monitored, the task execution mark is updated, and a task execution status report is generated.

[0052] See also Figure 2 , the graph building modules include:

[0053] Task metadata extraction submodule: extracts task metadata from the database, queries the database at the same time, obtains relevant information of all tasks, stores it in the data table, and generates a task metadata table;

[0054] Node mapping submodule: Based on the task metadata table, tasks and required resources are mapped to nodes in the graph. Each task corresponds to a task node, and each resource corresponds to a resource node. The task-resource node mapping table is generated by connecting the task nodes with the resource nodes.

[0055] Dependency building submodule: Based on the task resource node mapping table, the dependency relationships between tasks are obtained. Through the dependency identifiers between tasks, directed edges between tasks are established. The need for adding edges is determined based on the dependency strength. If the dependency strength is higher than the predetermined threshold, edge connections are added to generate a task resource dependency graph.

[0056] Task metadata extraction submodule: Based on the database, SQL query statements are used to extract task metadata from the database. Task-related fields, including task ID, execution duration, and resource requirements, are selected through SELECT statements. Query operations are performed to obtain relevant information about all tasks, which is stored in a temporary table. The extracted data is then stored in the task metadata table, which includes task ID, execution duration, and resource requirement information, generating tabular structured data.

[0057] Node mapping submodule: Based on the task metadata table, the node mapping method in graph theory is used to map tasks and required resources into nodes in the graph. Through the NetworkX library in Python, the add_node() function is used to create nodes for each task and resource. The task node includes the task ID, execution duration, and resource requirement information, and the resource node includes information such as resource type and available quantity. Then, the add_edge() function is used to connect the task node and resource node to generate a task-resource node mapping table.

[0058] Dependency construction submodule: Based on the task resource node mapping table, the Dijkstra algorithm is used to calculate the dependency between task nodes. Through the threshold judgment method of dependency strength, according to the dependency strength between tasks, if the dependency strength is higher than the preset threshold, a new dependency is added, and directed edges between tasks are constructed to form a task resource dependency graph.

[0059] See also Figure 2 , dependency resolution modules include:

[0060] Relationship extraction submodule: Based on the task resource dependency graph, it performs node traversal operations, reads the start and end node identifiers of each edge in the graph and compares them with the task numbers, obtains the corresponding relationship between task nodes, extracts the edge connection weights as the dependency strength values, constructs the dependency weight matrix, and generates the node dependency weight set;

[0061] Prioritization submodule: Based on the node dependency weight set, it performs a numerical comparison operation on the dependency strength. By summing the weight values ​​in the outgoing edge set of each node and sorting them, it extracts the strongly dependent nodes and establishes corresponding priority labels. It adjusts the priority arrangement in task scheduling and generates a task priority mapping table.

[0062] Execution Verification Submodule: Based on the task priority mapping table, it determines the executable status of each task node. By comparing the hierarchical position of the task in the topology structure and verifying whether all predecessor nodes have unfinished dependencies, it marks the executable and pending status, summarizes the sorting node and status information, and generates a task execution dependency list.

[0063] Relationship extraction submodule: Based on the task resource dependency graph, the graph traversal algorithm is used to traverse nodes through depth-first search. The start and end node identifiers of each edge in the graph are read and compared with the task numbers to obtain the corresponding relationship between task nodes. The connection weight of each edge is obtained, and the edge connection weight is extracted as the dependency strength value. The dependency weight matrix is ​​initialized and the corresponding values ​​in the matrix are filled according to the node dependency to generate the node dependency weight set.

[0064] Prioritization submodule: Based on the node dependency weight set, a sorting algorithm is used to sum the weight values ​​in the outgoing edge set of each node. Each weight in the outgoing edge set is cumulatively summed to obtain the total dependency strength value of each node. The dependency strength values ​​of each node are sorted in ascending or descending order, and nodes with higher dependency strength values ​​are extracted. Priority labels are established for the sorted nodes, and the nodes and their corresponding priorities are stored as a mapping relationship. The priority arrangement in task scheduling is adjusted to generate a task priority mapping table.

[0065] Execution verification submodule: Based on the task priority mapping table, a topological sorting algorithm is used to topologically sort the task nodes, determine the executable status of each task node, determine the in-degree of each node, and verify whether all predecessor nodes have unfinished dependencies. If the in-degree is 0 and all predecessor nodes have completed dependencies, it is marked as executable. If there are unfinished dependencies, it is marked as pending. The sorted nodes and status information are summarized to generate a task execution dependency list.

[0066] See also Figure 2 , the sequential prediction module includes:

[0067] Temporal structure analysis submodule: Based on the task execution dependency list, a long short-term memory neural network is used to extract the priority, dependency node number, and time window boundary corresponding to each task node. The module then performs time interval comparison operations, filters out overlapping interval tasks, calculates priority differences, establishes the time interaction structure of each task, and generates a time interaction matrix set.

[0068] Execution duration evaluation submodule: Based on the time interaction matrix set, the difference between the start and end time of each task is solved, and the actual duration of the task is determined in combination with the task resource usage density value. The task execution duration is updated and redistributed in intervals to generate a task interval duration table;

[0069] Scheduling sequence generation submodule: Based on the task interval duration table, it jointly sorts the priority, time span and required resources of all tasks, builds a scheduling starting point identification table based on time continuity and resource distribution sparsity, determines the order of task execution, establishes an executable process chain, and generates a task execution sequence table;

[0070] Temporal Structure Analysis Submodule: Based on the task execution dependency list, a long short-term memory neural network is used. The priority, dependency node number, and time window boundary corresponding to each task node are used as input features. The neural network model is constructed using the Keras library. The Adam optimizer is used with a learning rate of 0.001 for model training. The time-distributed function is used to compare the time intervals of each task node. The overlapping tasks are screened and the priority difference is calculated. The time interval overlap is determined, the time interaction structure of each task is established, and a time interaction matrix set is generated.

[0071] Execution duration evaluation submodule: Based on the time interaction matrix set, the time difference calculation method is used to calculate the difference between the start time and the end time of each group of tasks to obtain the duration of each task. Combined with the task resource usage density value, the resource usage density formula is used to calculate the actual duration of each task. The task execution duration is updated and redistributed in intervals, limiting the task execution time to a specified time range. After the update, a task interval duration table is generated;

[0072] Scheduling sequence generation submodule: Based on the task interval duration table, a sorting algorithm is used to jointly sort all tasks by priority, time span and required resources. During sorting, the weight of each task node is calculated through a custom sorting function, the time continuity of the task and the sparsity of resource distribution are calculated, the sparsity of resource distribution is evaluated, and a scheduling starting point identification table is constructed to determine the order of task execution, a directed graph is constructed, an executable process chain between tasks is established, and a task execution sequence table is generated.

[0073] Long short-term memory neural network, according to the formula:

[0074]

[0075] in: Indicates a task and tasks The priority gap between Indicates a task The start time, Indicates a task The end time, Indicates a task The start time, Indicates a task The end time, Indicates a task Priority, Indicates a task Priority, Indicates the weight coefficient of the priority difference calculation for the task time overlap. The weight coefficient that represents the impact of task priority differences on the calculation results, Indicates a task With the task The weight coefficient of task dependency on the calculation result, Indicates the weight coefficient of task start time difference to priority difference calculation, Indicates a task and tasks The task dependencies between

[0076] Execution process: Computing tasks and tasks The priority difference between the two tasks is first considered, the time overlap of the tasks is considered, and the tasks and tasks The temporal overlap is calculated by and To measure, the former means the task Start time and tasks The gap between the end times, the latter representing the task End time and tasks The interval between the start times, and then by introducing the weight coefficient , adjust the impact of time overlap, and make the role of total priority difference calculation more flexible and adjustable, then the priority difference of tasks is weighted and adjusted by the coefficient Considering the impact of priority differences on task scheduling, we further introduce dependencies between tasks. , through the coefficient Quantify the interdependence between tasks, reflecting the dependencies that must be followed during the scheduling process. Finally, the formula is calculated by Let's consider the task and tasks The difference in start time, coefficient It is used to adjust the weight of the difference in the total priority difference, comprehensively consider the time, priority and dependencies between tasks, and generate a more accurate scheduling priority difference.

[0077] See also Figure 2 , the resource allocation module includes:

[0078] Resource requirement extraction submodule: Based on the task execution sequence table, a graph neural network is used to extract task indexes and locate task numbers. By reading the resource type identifier and resource quantity value in the corresponding fields of the task, a number and value combination is constructed to match the relationship between the task and the required resources. The matching results are then registered in a unified record table to generate a task resource requirement list.

[0079] Allocation condition judgment submodule: Based on the task resource requirement list, it reads the system resource status snapshot and establishes a resource remaining quantity mapping table. It then calculates the numerical difference between the task required resources and the remaining resources, marks whether the current resources meet the task execution conditions, adds queue identifiers to insufficient resource items, and generates a resource allocation feasibility tag set.

[0080] Resource Prioritization Submodule: Based on the resource allocation feasibility tag set, it extracts all schedulable task numbers and corresponding priority values, divides the priority intervals and constructs resource level queues. It then maps the remaining system resources to each task group in proportion, registers the mapping relationship between tasks and resources, and generates a resource allocation optimization plan.

[0081] Resource requirement extraction submodule: Based on the task execution sequence table, a graph neural network is used to extract task indexes and locate task numbers. Graph data is loaded, task nodes are mapped to resource nodes, resource type identifiers and resource quantity values ​​in the corresponding fields of task nodes are read, resource requirement information of task nodes is extracted, number and value combinations are constructed, tasks are matched with required resources, matching results are recorded in a unified record table, and a task resource requirement list is generated.

[0082] Allocation condition judgment submodule: Based on the task resource requirement list, the resource status snapshot reading method is adopted. The current system resource status is read through SQL query statements. The remaining resource quantity is obtained from the resource status table using the SELECT statement. A resource remaining quantity mapping table is constructed. The numerical difference between the task required resources and the remaining resources is calculated in sequence. The difference between the task resource requirement and the remaining resources is calculated. It is marked whether the current resources meet the task execution conditions. The if-else statement is used to determine whether the resources are sufficient. If not, a queue identifier is added to the corresponding resource item to generate a resource allocation feasibility tag set.

[0083] Resource priority division submodule: Based on the resource allocation feasibility tag set, extract all schedulable task numbers and corresponding priority values, extract task numbers and priorities, divide the priority intervals and build resource level queues, divide the remaining resources into each task group in proportion, round off the resource allocation of each task to ensure the accuracy of resource allocation, register the mapping relationship between tasks and resources, and generate a resource allocation optimization plan.

[0084] Graph neural network, according to the formula:

[0085]

[0086] in: Indicates a task The total resource requirements calculated by the graph neural network, Resource Type The unit resource consumption value, For the task Resource Type The quantity of demand, is the number of all resource types, Resource Type Task The weight coefficient of resource demand, For the task and resource types The dependence between is the adjustment weight coefficient of time resource consumption to total resource demand, To calculate the adjustment weight coefficient of resource consumption to total resource demand, For the task Time resource requirements, For the task Computing resource requirements;

[0087] Execution process: First, according to the resource type and tasks Required quantity Calculate the consumption value of each resource, Indicates resource type The unit resource consumption value, then introduce the weight coefficient , which aims to adjust the different types of resources to tasks The impact of resource requirements reflects the importance of each resource type, and then the task and resources Dependence between Further adjust the calculation. The higher the dependency, the stronger the task's demand for resources, which affects resource consumption. Next, consider the time resource demand. and computing resource requirements , respectively represent the time and computing power resources required for task execution, and the weight coefficient and Further adjust the proportion of resources in the total demand to ensure that the time and computing resource requirements of the task are properly considered, and finally obtain the task through comprehensive calculation. Total resource requirements , providing optimized resource allocation basis for the task scheduling system.

[0088] See also Figure 2 , the execution control module includes:

[0089] Task trigger judgment submodule: Based on the resource allocation optimization plan, it performs a joint query on the task and resource locking status. By performing an availability judgment operation on the status field of the task-bound resource, it marks the tasks with unoccupied resources as pending and creates a numbered list to generate a task activation identification set.

[0090] Status record execution submodule: Based on the task activation identifier set, it traverses all pending task numbers, obtains the execution start time and resource usage status, records the task execution time period and writes it into the task status field, sets the resource to be occupied, constructs a mapping relationship between the number and time, and generates a task execution status record table;

[0091] Execution result update submodule: Based on the task execution status record table, poll to obtain the current execution status value of the task, determine whether the task completion mark and the upper limit of the execution time meet the termination conditions, release the occupied resources and update the execution status code and task completion time, complete the status writing operation, and generate a task execution status report;

[0092] Task trigger judgment submodule: Based on the resource allocation optimization plan, SQL query statements are used to query the task and resource locking status. The resource occupancy status field is extracted from the resource status table through the SELECT statement, and the availability judgment operation is performed. The status of the task-bound resource is judged using the if-else statement. If the resource is not occupied, the task is marked as pending, and the number of the pending task is added to the number list to generate the task activation identification set.

[0093] Status record execution submodule: Based on the task activation identifier set, a for loop is used to traverse all pending task numbers, SQL query statements are used to read the task execution start time and resource usage status, and a task execution status data table is constructed. The task execution time period is recorded and the task status field is updated. The resource is set to the occupied state, a mapping relationship between the number and time is constructed, and a task execution status record table is generated.

[0094] Execution result update submodule: Based on the task execution status record table, a polling algorithm is used to regularly obtain the current execution status value of the task, obtain the task completion mark and execution duration information, and use the if-else statement to determine whether the task completion mark and the upper limit of the execution duration meet the termination conditions. If the conditions are met, the SQLUPDATE statement is used to release the occupied resources and update the execution status code and task completion time. The UPDATE operation is executed to update the status field of the task, the status write operation is completed, and the task execution status report is generated.

[0095] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A log analysis task scheduling system based on neural network, characterized in that: The system comprises: Graph construction module: This module extracts task metadata from the database, obtains task execution duration and resource requirements, maps tasks and resource nodes, establishes inter-task dependency relationships as edges, determines whether to add new edges based on dependency strength, and uses priority and resource quota as node attributes to construct a task resource dependency graph. Dependency parsing module: Based on the task resource dependency graph, obtains the dependency relationship between task nodes, calculates the dependency matrix, sorts the nodes by dependency strength, adjusts the priority if the dependency strength exceeds a threshold, determines whether the task can be executed, and generates a task execution dependency list; Sequence prediction module: Based on the task execution dependency list, a long short-term memory neural network is used to analyze task priorities and time windows, estimate execution time, sort the task execution order by priority, and generate a task execution sequence table based on task delays and resource requirements; Resource allocation module: Based on the task execution sequence table, a graph neural network is used to extract the resource requirements of each task, compare the resource status, determine whether there are sufficient resources to support concurrent execution, adjust resource allocation according to priority, suspend the execution of some tasks if resources are insufficient, and generate a resource allocation optimization plan; Execution control module: Based on the resource allocation optimization plan, it determines whether the resources meet the task execution requirements. If available, it starts the task execution and registers the status. If insufficient, it pauses the task and reschedules it. It monitors the task status changes, updates the execution mark, and generates a task execution status report. The sequence prediction module includes: Temporal structure analysis submodule: Based on the task execution dependency list, a long short-term memory neural network is used to extract the priority, dependency node number, and time window boundary corresponding to each task node, perform time interval comparison operations, screen overlapping interval tasks, calculate priority differences, establish the time interaction structure of each task, and generate a time interaction matrix set; Execution duration evaluation submodule: Based on the time interaction matrix set, the difference between the start time and the end time of each task is solved, and the actual duration of the task is determined in combination with the task resource usage density value, and the task execution duration is updated and redistributed in intervals to generate a task interval duration table; Scheduling sequence generation submodule: Based on the task interval duration table, the priority, time span and required resources of all tasks are jointly sorted, and a scheduling starting point identification table is constructed through time continuity and resource distribution sparsity to determine the order of task execution, establish an executable process chain, and generate a task execution sequence table; The long short-term memory neural network is based on the formula: in: Indicates a task and tasks The priority gap between Indicates a task The start time, Indicates a task The end time, Indicates a task The start time, express The end time of the task, Indicates a task Priority, Indicates a task Priority, Indicates the weight coefficient of the priority difference calculation for the task time overlap. The weight coefficient that represents the impact of task priority differences on the calculation results, Indicates a task With the task The weight coefficient of task dependency on the calculation result, Indicates the weight coefficient of task start time difference to priority difference calculation, Indicates a task and tasks The dependency between tasks.

2. The neural network-based log analysis task scheduling system according to claim 1, characterized in that: The graph construction module includes: Task metadata extraction submodule: extracts task metadata from the database, queries the database at the same time, obtains relevant information of all tasks, stores it in the data table, and generates a task metadata table; Node mapping submodule: Based on the task metadata table, tasks and required resources are mapped into nodes in the graph, each task corresponds to a task node, each resource corresponds to a resource node, and a task-resource node mapping table is generated by connecting the task nodes with the resource nodes; Dependency construction submodule: Based on the task resource node mapping table, the dependency relationship between tasks is obtained, and directed edges between tasks are established through the dependency identification between tasks. It is determined whether new edges need to be added according to the dependency strength. If the dependency strength is higher than the predetermined threshold, the edge connection is added to generate a task resource dependency graph.

3. The neural network-based log analysis task scheduling system according to claim 1, characterized in that: The dependency parsing module includes: Relationship extraction submodule: Based on the task resource dependency graph, perform node traversal operations, read the start and end node identifiers of each edge in the graph and compare them with the task numbers, obtain the corresponding relationship between task nodes, extract the edge connection weight as the dependency strength value, construct the dependency weight matrix, and generate the node dependency weight set; Prioritization submodule: Based on the node dependency weight set, a numerical comparison operation of dependency strength is performed. By summing the weight values ​​in the outgoing edge set of each node and sorting them by size, the strongly dependent nodes are extracted and corresponding priority labels are established. The priority arrangement in task scheduling is adjusted to generate a task priority mapping table. Execution verification submodule: Based on the task priority mapping table, determine the executable status of each task node, compare the hierarchical position of the task in the topology structure and verify whether all predecessor nodes have unfinished dependencies, mark the executable and pending states, summarize the sorting nodes and status information, and generate a task execution dependency list.

4. The neural network-based log analysis task scheduling system according to claim 1, characterized in that: The resource allocation module includes: Resource requirement extraction submodule: Based on the task execution sequence table, a graph neural network is used to extract task indexes and locate task numbers. By reading the resource type identifier and resource quantity value in the corresponding field of the task, a number and value combination is constructed to match the relationship between the task and the required resources. The matching results are then registered in a unified record table to generate a task resource requirement list. Allocation condition judgment submodule: Based on the task resource requirement list, the system resource status snapshot is read and a resource remaining quantity mapping table is established. The numerical difference between the task required resources and the remaining resources is calculated in sequence. The current resources are marked as meeting the task execution conditions. A queue mark is added to the insufficient resource items to generate a resource allocation feasibility mark set. Resource priority division submodule: Based on the resource allocation feasibility tag set, extract all schedulable task numbers and corresponding priority values, divide the priority intervals and build a resource level queue, map the remaining system resources to each task group in proportion and register the mapping relationship between tasks and resources, and generate a resource allocation optimization plan.

5. The neural network-based log analysis task scheduling system according to claim 4 is characterized in that: The graph neural network is based on the formula: in: Indicates a task The total resource requirements calculated by the graph neural network, Resource Type The unit resource consumption value, For the task Resource Type The quantity of demand, is the number of all resource types, Resource Type Task The weight coefficient of resource demand, For the task and resource types The dependence between is the adjustment weight coefficient of time resource consumption to total resource demand, To calculate the adjustment weight coefficient of resource consumption to total resource demand, For the task Time resource requirements, For the task computing resource requirements.

6. The neural network-based log analysis task scheduling system according to claim 1, characterized in that: The execution control module includes: Task trigger judgment submodule: Based on the resource allocation optimization plan, it performs a joint query on the task and resource locking status, performs an availability judgment operation on the status field of the task-bound resource, marks the task in the non-occupied resource state as pending, creates a numbered list, and generates a task activation identification set; Status record execution submodule: based on the task activation identification set, traverses all task numbers to be executed, obtains the execution start time and resource usage status, records the task execution time period and writes it into the task status field, sets the resource to be occupied, constructs a mapping relationship between the number and time, and generates a task execution status record table; Execution result update submodule: Based on the task execution status record table, poll to obtain the current execution status value of the task, determine whether the task completion mark and the upper limit of the execution time meet the termination conditions, release the occupied resources and update the execution status code and task completion time, complete the status write operation, and generate a task execution status report.

Images