Shell script task scheduling method and system in network-free state

Abstract

The invention discloses a Shell script task scheduling method and system in a network-free state, and relates to the technical field of information, and the method comprises the steps: in the network-free state, according to the priority of each Shell script task to be scheduled and executed, sorting each Shell script task according to the priority, and forming a task priority queue; if the preset triggering condition is met, extracting a target Shell script task with the highest priority from the task priority queue, and submitting the target Shell script task to an execution queue; executing the target Shell script task in a network-free state; detecting execution parameters in the process of executing the target Shell script task by adopting an anomaly detection algorithm; and if an exception is detected, performing exception handling on the target Shell script task by adopting an exception handling strategy. According to the method and the device, the Shell script task can be effectively executed in a network-free state.

Description

Technical Field

[0001] The invention relates to the field of information technology, and in particular to a method and system for scheduling Shell script tasks in a non-network state. Background Art

[0002] Shell scripts are mainly used to automatically execute a series of commands and tasks, as well as task scheduling, manage system resources, and perform system management tasks. When performing task scheduling, Shell scripts are usually executed in a network environment. When performing Shell script task scheduling without a network, there are technical challenges in terms of task triggering mechanism, execution sequence control, and abnormal recovery.

[0003] Since it is impossible to rely on the network environment for centralized scheduling, the triggering of Shell script tasks needs to rely entirely on local time or events. However, in complex business scenarios, there may be dependencies and priority constraints between Shell script tasks, which results in Shell script tasks not being able to be executed in the expected order in an environment without a network. When task exceptions occur during execution, there is no corresponding processing mechanism, and it cannot guarantee that the task can be completed smoothly or terminated in time. In addition, in a long offline state, Shell script tasks may continue to accumulate. Once the network connection is restored, the accumulated tasks cannot be processed, and it is easy to repeat or miss tasks, which cannot guarantee the continuity of tasks, and it is not compatible with synchronizing the task status when there is no network connection and after the network connection is restored.

[0004] In summary, the Shell script task scheduling method in the prior art cannot effectively perform task scheduling in a non-network state, and cannot ensure effective execution of tasks. Summary of the invention

[0005] In order to solve the problem that the Shell script task scheduling method in the prior art cannot effectively perform task scheduling in a non-network state and cannot ensure the effective execution of tasks.

[0006] The purpose of the present invention is to adopt the following technical solutions to achieve: In a first aspect, the present invention proposes a method for scheduling Shell script tasks in a non-network state, the method comprising: When in a no-network state, according to the priority of each Shell script task to be scheduled for execution, each Shell script task is sorted according to the priority to form a task priority queue; If the preset triggering condition is met, the target Shell script task with the highest priority is extracted from the task priority queue, and the target Shell script task is submitted to the execution queue; Executing the target Shell script task in the execution queue in a no-network state; An exception detection algorithm is used to detect execution parameters during the execution of the target Shell script task; if an exception is detected, an exception handling strategy is used to perform exception handling on the target Shell script task.

[0007] Preferably, according to the priority of each Shell script task to be scheduled for execution, each Shell script task is sorted according to the priority to form a task priority queue, including: Obtain the dependency of each Shell script task to be scheduled and executed, and determine the dependency relationship between every two Shell script tasks; According to the dependency relationship between each two Shell script tasks, each Shell script task is grouped to form a plurality of task sub-queues; Draw a directed graph for the Shell script tasks in each task subqueue; Performing topological sorting according to the directed graph to determine the execution order of the Shell script tasks in each task subqueue; Based on each task sub-queue, determining the highest priority in the execution order of the Shell script tasks in the task sub-queue as the priority of the task sub-queue; Sort multiple task subqueues by priority to form a task priority queue.

[0008] Preferably, the use of an anomaly detection algorithm to detect execution parameters during the execution of the target Shell script task includes: Obtaining the execution log of the target Shell script task and the data of system resource occupation, preprocessing the data, and extracting key features; Performing anomaly detection on the key features using an anomaly detection algorithm; If an exception is detected, the exception information is recorded and the state of the target Shell script task is marked as abnormal.

[0009] Further, the adopting of the exception handling strategy to perform exception handling on the target Shell script task includes: Acquire the exception type and task importance attributes of the target Shell script task in the exception information; Determine whether to retry executing the target Shell script task according to the exception type and task importance; If yes, the task retry mechanism is triggered to re-execute the target Shell script task and update the status information in the execution log; If not, the target Shell script task is skipped, and the skipping reason and status information are recorded in the execution log.

[0010] Furthermore, before executing the target Shell script task in the execution queue in the no-network state, the method further includes: Obtaining task metadata and task parameters of the target Shell script task; Performing data compression processing on the task metadata and the task parameters to obtain compressed data; The compressed data is processed into blocks using distributed storage to obtain task data blocks.

[0011] Furthermore, after the task retry mechanism is triggered, it also includes: The compressed data is read from the task data block, and the target Shell script task is restored to a waiting state.

[0012] Preferably, after executing the target Shell script task in the execution queue in a non-network state, the method further comprises: When the network connection is restored, the priority and execution status of each Shell script task to be scheduled for execution are obtained; A batch processing strategy is adopted according to the priority and execution status of each Shell script task, and the execution data of each Shell script task is gradually reported to the central node, and data synchronization is performed through a handshake protocol with the central node.

[0013] In a second aspect, the present invention proposes a Shell script task scheduling system in a non-network state, comprising: The task queue module is used to sort each Shell script task according to the priority of each Shell script task to be scheduled for execution, so as to form a task priority queue; The task trigger module is used to submit the target Shell script task that meets the trigger condition to the execution queue; A task execution module, used for executing the target Shell script task in the execution queue in a non-network state; An anomaly detection module, used for performing anomaly detection on execution parameters during execution of the target Shell script task; The exception handling module is used to perform exception handling on the target Shell script task for which an exception detection result is detected.

[0014] Preferably, the system further comprises: The task data storage module is used to compress the task metadata and task parameters in the Shell script task to obtain compressed data before the target Shell script task is executed, perform block processing on the compressed data using distributed storage, and store the task data blocks after the block processing.

[0015] Preferably, the system further comprises: The network synchronization module is used to obtain the priority and execution status of each Shell script task to be scheduled for execution, adopt a batch processing strategy, gradually report the execution data of each Shell script task to the central node, and synchronize data through a handshake protocol with the central node.

[0016] In a third aspect, the present invention provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the Shell script task scheduling method in a non-network state as described above is implemented.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for scheduling Shell script tasks in a network-free state as described above is implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method and system for scheduling Shell script tasks in a network-free state. The method obtains the priority of each Shell script task to be scheduled and executed, forms a task priority queue after sorting, and triggers the task execution by an event-driven mechanism according to the triggering condition of the Shell script task, thereby ensuring that the high-priority Shell script task can be executed first. By using an anomaly detection algorithm to perform anomaly monitoring during the execution process, anomalies in the task execution process are discovered in time, and the abnormal Shell script task is processed according to the anomaly processing strategy, thereby ensuring that the Shell script task can be effectively scheduled and executed in a network-free state, avoiding task accumulation and execution conflicts, thereby improving the stability and reliability of task execution, and ensuring the long-term stable operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a Shell script task scheduling method in a non-network state in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the process of establishing a priority queue in Example 1 of the present invention; Figure 3 This is a schematic diagram of the Shell script task anomaly detection process in Example 1 of the present invention; Figure 4 The figure is a schematic diagram of the Shell script task exception handling process in Example 1 of the present invention; Figure 5 This is a schematic diagram of the structure of a Shell script task scheduling system in a non-network state in Example 2 of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device in Example 3 of the present invention. DETAILED DESCRIPTION

[0020] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0021] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0022] Embodiment 1: A Shell script task scheduling method in a non-network state in an embodiment of the present invention, the flow chart is as follows Figure 1 As shown, including: When in a no-network state, according to the priority of each Shell script task to be scheduled for execution, each Shell script task is sorted according to the priority to form a task priority queue; If the preset triggering condition is met, the target Shell script task with the highest priority is extracted from the task priority queue, and the target Shell script task is submitted to the execution queue; Executing the target Shell script task in the execution queue in a no-network state; An exception detection algorithm is used to detect execution parameters during the execution of the target Shell script task; if an exception detection result is detected, an exception handling strategy is used to perform exception handling on the target Shell script task.

[0023] Specifically, Figure 2 As shown, in the embodiment of the present invention, according to the priority of each Shell script task to be scheduled for execution, each Shell script task is sorted according to the priority to form a task priority queue, including: Obtain the dependency of each Shell script task to be scheduled and executed, and determine the dependency relationship between every two Shell script tasks; According to the dependency relationship between each two Shell script tasks, each Shell script task is grouped to form a plurality of task sub-queues; Draw a directed graph for the Shell script tasks in each task subqueue; Performing topological sorting according to the directed graph to determine the execution order of the Shell script tasks in each task subqueue; Based on each task sub-queue, determining the highest priority in the execution order of the Shell script tasks in the task sub-queue as the priority of the task sub-queue; Sort multiple task subqueues by priority to form a task priority queue.

[0024] Specifically, the dependency of each Shell script task to be scheduled is obtained, and the dependency relationship between each two Shell script tasks is determined; exemplary: the system first obtains the task source priority and task source dependency, for example, task A has a high priority and is dependent on task B; task B has a medium priority and has no dependency. The system first determines the dependency relationship between each two Shell script tasks.

[0025] Then, according to the dependency relationship between each two Shell script tasks, each Shell script task is grouped to form a plurality of task sub-queues (ie, a task scheduling sequence including Task B and Task A).

[0026] Draw a directed graph for the Shell script tasks in each task subqueue; perform topological sorting according to the directed graph to determine the execution order of the Shell script tasks in each task subqueue; wherein the nodes in the directed graph represent tasks, and the directed edges represent the dependency relationship between tasks, that is, the starting task of the edge needs to be executed before the end task of the edge.

[0027] Calculate the in-degree of each node in the directed graph, that is, the number of edges pointing to the node. A node with an in-degree of 0 means that no other tasks depend on it, so it can be executed first.

[0028] According to the directed graph, a topological sorting algorithm can be used to determine the execution order of tasks, including: adding all nodes with in-degree 0 to a set.

[0029] Take a node from the set, add it to the result list, and remove the node and all of its outgoing edges (that is, other nodes that the node points to) from the graph.

[0030] Update the in-degree of the affected nodes (i.e., those nodes pointed to by the deleted edges). If the in-degree of a node becomes 0, add it to the set.

[0031] Repeat the above process until there are no nodes left in the graph.

[0032] If all nodes cannot be added to the result list during topological sorting, it means that there is a cycle in the graph. This indicates that there are circular dependencies between tasks and the task dependencies need to be rechecked and adjusted.

[0033] Based on the result of topological sorting, we can get a task execution order. Executing tasks in this order can ensure that all dependencies are met. That is, the system removes interdependent tasks through an algorithm and obtains a task scheduling sequence sorted by priority (that is, a task scheduling sequence sorted by priority while removing interdependencies is obtained).

[0034] The priority of each sorted task subqueue is set as the highest priority of the Shell script task in the execution order; and the multiple task subqueues with reset priorities are sorted according to the priorities to form a task priority queue.

[0035] In the embodiments of the present invention, priority and dependency management are used to ensure that key tasks are executed first and avoid waste of resources. For example, in actual business, the data analysis tasks of the e-commerce platform have a higher priority and are dependent on the data collection tasks. If the data analysis tasks are seriously backlogged, the system will prioritize the data collection tasks to ensure the smooth execution of the data analysis tasks. For another example, the transaction processing tasks of the financial system have a high priority and are dependent on the account verification tasks. If the transaction processing tasks are backlogged, the system will prioritize the account verification tasks to ensure that the transactions are completed in a timely manner. This not only improves transaction efficiency, but also reduces financial risks caused by delays. The embodiments of the present invention can significantly improve the system operation efficiency, ensure that key tasks are completed on time, optimize user experience, reduce business risks, and can flexibly respond to various complex situations to ensure the stability and reliability of task execution through reasonable task scheduling and resource allocation strategies.

[0036] Specifically, if the preset trigger condition is met, the target Shell script task with the highest priority is extracted from the task priority queue, and the target Shell script task is submitted to the execution queue; illustratively, in the embodiment of the present invention, for each Shell script task, according to its priority and the preset trigger condition, an event-driven mechanism is used to trigger task execution, including: monitoring system events, judging whether to trigger task execution according to the preset trigger condition. If the trigger condition is met, the task with the highest priority is taken out from the priority queue. The execution parameters and environment dependencies of the task are obtained, computing resources are allocated through the task scheduler, and the task is submitted to the execution queue.

[0037] Specifically, the task management system organizes and schedules tasks through priority queues. Each task has a preset priority and trigger condition. For example, the system continuously monitors various events, such as the arrival of new data, changes in system load, etc., and determines whether to execute the task based on the preset trigger conditions. When the conditions are met, the system takes the highest priority task from the priority queue. For example, when new user behavior data is detected, a user portrait update task may be triggered. Before the task is executed, the system needs to prepare the corresponding execution environment and parameters. For machine learning tasks, this may include loading training data, setting model parameters, etc. The task scheduler is responsible for allocating computing resources, such as allocating GPU resources for image processing tasks and allocating large memory nodes for data analysis tasks.

[0038] Specifically, in an embodiment of the present invention, executing the target Shell script task in the execution queue in a non-network state includes: The execution queue schedules tasks to be executed on the computing nodes according to the task priority and submission order. The task status and progress are recorded during the execution process. After the task is completed, the execution result and log are obtained, the task status is updated to completed, and the task is removed from the priority queue and the execution queue.

[0039] Specifically, the execution queue arranges tasks to run on computing nodes according to task priority and submission order. The system records task status and progress in real time for easy monitoring and management. For example, for long-running batch tasks, the system may record the amount of data processed and the time consumed at regular intervals. After the task is completed, the system obtains the execution results and logs, updates the task status to completed, and removes the task from the relevant queue.

[0040] Specifically, in the embodiment of the present invention, an exception detection algorithm is used to detect the execution parameters during the execution of the target Shell script task; if an exception is detected, an exception handling strategy is used to handle the exception of the target Shell script task. Figure 3 As shown, specifically including: Obtain the execution log of the target Shell script task and the data of system resource usage, preprocess the data, and extract key features. Use anomaly detection algorithms, such as isolation forest or single-class SVM, to perform anomaly detection on the extracted key features to determine whether there is task blocking or resource usage anomaly.

[0041] If an exception is detected, the exception information is recorded and the state of the target Shell script task is marked as abnormal.

[0042] Specifically, obtaining task execution logs and system resource usage data is the basis for monitoring the running status of tasks. For example, the background task system of an e-commerce platform records the logs of each order processing task, including the task start time, end time, execution time, etc., and monitors the system resource usage, such as CPU usage, memory usage, disk I / O, etc. These data are collected through log collection tools such as Fluentd or ELKStack and stored in a distributed database. The collected data is preprocessed, including data cleaning, formatting, and standardization. Suppose a task execution log shows that the CPU usage suddenly soared to more than 90% within a certain period of time, while it should be around 50% under normal circumstances, which indicates that there may be anomalies. The preprocessing stage will mark these abnormal data for subsequent analysis. Extracting key features is a key step in anomaly detection.

[0043] In the embodiment of the present invention, the isolation forest algorithm is taken as an example. The algorithm isolates data points by constructing multiple decision trees, and finally determines whether it is an abnormal point based on the depth and number of times the data point is isolated. Assuming that the extracted features include task execution time, CPU usage, memory usage, etc., the isolation forest algorithm can identify those abnormal points that deviate significantly from the normal range. If an abnormality is detected, such as a task execution time far exceeding the average value, the system will trigger the task termination mechanism, record the abnormal information, and mark the Shell script task status as abnormal. Among them, the abnormal information includes the abnormal type (such as "execution timeout"), the time of occurrence and other information, which are stored in the abnormal log database for subsequent analysis and processing.

[0044] Specifically, in the embodiment of the present invention, an exception handling strategy is used to handle the exception of the target Shell script task, such as Figure 4 As shown, including: Get the exception type and task importance attributes. According to the preset exception handling strategy, combined with the exception type and task importance, determine whether the task needs to be retried. If retry is required, the task retry mechanism is triggered, the task is re-executed, and the relevant status information in the task execution log is updated. If retry is not required or the number of retries reaches the upper limit, the task is skipped, and the skipping reason and related information are recorded in the task execution log.

[0045] According to the status information recorded in the execution log, the final execution status of the task is updated, such as execution success, execution failure, skip, etc. The task execution log is persistently stored for subsequent task execution analysis and optimization, and the exception handling and retry strategies of task execution are continuously improved.

[0046] Specifically, in the embodiments of the present invention, different processing levels are set for different exception types and task importance. For example, for a real-time data stream processing task, if a short network fluctuation causes data loss, you can choose to ignore it and continue processing, because real-time is more important. For a batch processing task of financial transaction data, even a small data loss may cause serious consequences, so it is necessary to immediately stop the task and re-execute it or perform manual processing.

[0047] The task retry mechanism is an effective way to handle temporary failures. For example, in a distributed computing environment, a computing node may be temporarily unresponsive due to excessive load. At this point, the system can reallocate the task to other nodes for execution. An exponential backoff strategy can be used when retrying, that is, the interval between each retry gradually increases to avoid additional pressure on the system. For tasks that need to be skipped, it is very important to record the reason for the skip. For example, in a data cleaning task, if a piece of data has a serious format error and cannot be processed, the system will skip the data, but will record the data content and the cause of the error in detail in the log for subsequent manual analysis and processing.

[0048] The task execution log is an important data structure that records key information during the task execution process. Taking a data processing task as an example, the log may contain the task start time, the amount of data processed, the processing speed, resource usage, etc. This information helps to analyze task performance and optimize execution strategies. For example, it is possible to use a decision tree to generate a recovery strategy based on the task exception type and historical data. For example, if a task fails due to a memory overflow, the decision tree will use strategies such as "increase memory allocation" or "optimize code to reduce memory usage" during the re-execution process based on historical data. After executing the task recovery strategy, the system will monitor the task recovery process. If the recovery fails, such as memory overflow still occurs after restarting the task, the system will record the failure information and trigger the manual processing process to notify the operation and maintenance personnel to intervene in the investigation.

[0049] Specifically, in the embodiment of the present invention, before executing the target Shell script task in the execution queue in a network-free state, the method further includes: Obtaining task metadata and task parameters of the target Shell script task; Performing data compression processing on the task metadata and the task parameters to obtain compressed data; The compressed data is processed into blocks using distributed storage to obtain task data blocks.

[0050] Specifically, the data acquisition module is used to collect the task metadata and task parameters of the Shell script task; the task metadata and task parameters are obtained, and the data compression module is used to perform data compression processing to obtain compressed data. The encrypted compressed data is obtained through data encryption processing, and the local file data address is obtained. The encrypted compressed data is divided into blocks using distributed storage to obtain task data blocks, and the task data blocks are stored in the local file address. When an abnormal situation occurs, according to the task metadata information of the Shell script task, the data reading module is used to restore the task to the waiting state through the exception handling module. The task metadata and task parameters of the Shell script task are the data before the task is executed.

[0051] Exemplarily, in an offline task processing system, the data acquisition module first collects metadata and parameters of the task. Task metadata includes information such as the type of task, creation time, priority tag, and dependency. For example, the metadata of a video processing task may include video format, resolution, priority tag "high", and dependency on the preceding audio processing task. For example, the parameters of a video processing task may include encoding method, bit rate, etc. These data are compressed by the data compression module to reduce the storage and transmission burden. Assuming that the original video data is 1GB, it may be reduced to 500MB after compression. After compressing the data, the system obtains data write interface information and determines the storage location type. Assuming that the storage location is a local file, the system obtains the local file data address, such as " / data / videos / processed / ". Through distributed storage technology, the system divides the encrypted compressed data into blocks. Assuming that the size of each data block is 100MB, 1GB of video data will be divided into 10 data blocks. These data blocks are stored in the specified local file address to ensure data security and recoverability.

[0052] Further, after the task retry mechanism is triggered, the compressed data is read from the task data block, and the target Shell script task is restored to a waiting state.

[0053] Exemplarily, in abnormal situations, such as system crashes or network interruptions, the system obtains the task execution status data at the time of the abnormality through the data reading module based on the task metadata information. For example, an exception occurs in the video processing task when processing the 5th data block, and the system records the status at this time. Through the exception handling module, the system restores the task execution to the state before the exception, ensuring the continuity of the task and the integrity of the data. The use of data compression and encryption processing in the embodiments of the present invention not only improves storage efficiency, but also enhances data security. The application of distributed storage and transaction mechanisms ensures high availability and consistency of data. The exception handling mechanism provides powerful fault tolerance and ensures the stable operation of the system. The present invention can restore the execution status of the task even in abnormal situations by persisting the metadata and parameters of the task in a local file or database.

[0054] Specifically, in the embodiment of the present invention, after the network connection is restored, a batch processing strategy is adopted according to the priority and execution status of the task, and the locally accumulated task results are gradually reported to the central node. Through the handshake protocol with the central node, it is ensured that the local task status is consistent with the central node to avoid repeated execution of tasks.

[0055] Specifically, the obtaining of the priority and execution status of each Shell script task to be scheduled for execution, adopting a batch processing strategy according to the priority and the execution status, gradually reporting the locally accumulated task results to the central node, and synchronizing the local task status with the central node through a handshake protocol with the central node, includes: Establishing a task set for each Shell script task to be scheduled for execution; Obtaining the priority of each Shell script task in the task set, and arranging each task in descending order according to the priority value to form an ordered task array; According to the task status of each Shell script task in the ordered task array, the tasks are classified and merged according to the task status type to obtain a plurality of task sets of different status types; Divide the plurality of task sets of different status types equally by setting a set number value to obtain corresponding subset groups; Generating handshake packets to be sent in batches from the execution data of each Shell script task in the subset group, and sending the handshake packets to the central node server; The central node server determines the execution result of the corresponding task according to the task data in the handshake packet, updates the task status on the network and sends a confirmation instruction of the completion of the execution, so as to synchronize the local task status with the central node.

[0056] The execution data of each Shell script task includes data of the task execution process and execution result.

[0057] Exemplarily, the priority of each task in the local task set is obtained, and each task is arranged in descending order according to the priority value to form a first sequence. For the first sequence, tasks are extracted in order from high to low using the priority value to form an ordered task array. The number of tasks in the ordered task array is determined. According to the status of each task in the ordered task array, they are classified and merged according to the status type to obtain a task set containing different status types. Using a timestamp, for different status type task sets, the most recent status update time of the tasks in each set is recorded. The maximum time value in each set is obtained. All status type task sets are arranged in ascending order according to the maximum time value. Multiple status type task sets are equally divided by setting a value for the number of sets to obtain corresponding sub-set groups.

[0058] Generate handshake packets to be sent by batching the data of each subset. Send the handshake packet to the central node server. If the central node server replies with a response packet, the task number group in the response packet is obtained. Determine whether the tasks in the task number group have been completed. Obtain the unfinished tasks in the task number group and build the corresponding instruction set according to the handshake protocol. Use the local counter to generate the time value to mark the timestamp of the unfinished task instruction set, and synchronize it to the central node server for instruction set registration. Obtain confirmation that the central node instruction set has been executed. According to the confirmation result of the execution of the central node instruction set, determine the task group number corresponding to the sent data group through the sending status bit of each group, determine the sent confirmation data group according to the confirmation status bit of each group, and obtain the total task volume by traversing the counter value corresponding to each task group number. Determine the difference between the total task volume and the number of task numbers in the instruction set.

[0059] For example, task priority management is a key link in distributed systems. By prioritizing local task sets, important tasks can be ensured to be processed in a timely manner. For example, in an e-commerce platform, tasks such as order processing, inventory update, and user authentication may have different priorities. Arranging these tasks in descending order of priority to form an ordered task array helps to reasonably allocate system resources. Task status classification is an important means of monitoring system operation. By classifying tasks according to states such as "running", "waiting", and "completed", you can quickly understand the overall operation of the system. By recording the last update time of each state type task set, you can identify tasks that have not been updated for a long time and discover potential problems in time. Timestamps play a vital role in distributed systems. By arranging different state type task sets in ascending order according to the maximum time value, you can prioritize the task sets that have not been updated for a long time and improve the system response speed. Dividing the sorted set into subset groups can achieve batch processing of tasks and improve system throughput.

[0060] The handshake packet mechanism is an important means to ensure the reliability of communication. The processed data is sent to the central node server in batches to generate handshake packets, which can achieve reliable data transmission. By receiving the response packet from the server, it is possible to confirm whether the data is successfully received and obtain the task number that needs further processing. Task execution status tracking is the key to ensuring system stability. By judging the execution status of tasks in the task number group, unfinished tasks can be discovered in time. For these tasks, the system will build the corresponding instruction set according to the predefined handshake protocol and use the local counter to generate a timestamp to ensure the timeliness of the instructions. The instruction set registration and execution confirmation mechanism can ensure the reliable execution of tasks. Synchronizing the instruction set with a timestamp to the central node server for registration and waiting for confirmation of completion can effectively prevent the loss or duplication of tasks. This mechanism is particularly important in distributed systems and can ensure data consistency and integrity. Task completion evaluation is the basis for system performance optimization. By comparing the total task volume and the number of confirmed completed tasks, the system's task completion rate can be obtained. This indicator can be used to evaluate the efficiency of the system, discover potential performance bottlenecks, and provide a basis for subsequent system optimization.

[0061] The present invention provides a method for scheduling Shell script tasks in a network-free state, which is used to solve the problem of how to ensure that preset Shell script tasks can be executed in order according to the priority and preset execution time in a network-free state, and synchronize with the remote task management server after the network is restored. The method obtains the priority of each Shell script task to be scheduled and executed, forms a task priority queue after sorting, and triggers the task execution according to the triggering condition of the Shell script task by using an event-driven mechanism to ensure that high-priority Shell script tasks can be executed first. By using an anomaly detection algorithm to perform anomaly monitoring during the execution process, anomalies in the task execution process are discovered in time, and the abnormal Shell script tasks are processed according to the anomaly handling strategy, it is ensured that the Shell script tasks can be effectively scheduled and executed in a network-free state, and task accumulation and execution conflicts are avoided, thereby improving the stability and reliability of task execution and ensuring the long-term stable operation of the system.

[0062] The present invention also has an intelligent task recovery function. After the network is restored, the local task results are reported to the central node in batches to ensure the consistency of the status, realizing the automatic scheduling and execution of Shell scripts in a network-free environment, and improving the reliability and efficiency of the system. The present invention can be integrated with the existing task scheduler, has high compatibility, and is particularly suitable for scenarios where automated tasks need to be executed in a network-off environment.

[0063] Embodiment 2: The present invention based on the same inventive concept also provides a Shell script task scheduling system in a non-network state, such as Figure 5 As shown, including: The task queue module is used to sort each Shell script task according to the priority of each Shell script task to be scheduled for execution, so as to form a task priority queue; The task trigger module is used to submit the target Shell script task that meets the trigger condition to the execution queue; A task execution module, used for executing the target Shell script task in the execution queue in a non-network state; An anomaly detection module, used for performing anomaly detection on execution parameters during execution of the target Shell script task; The exception handling module is used to perform exception handling on the target Shell script task for which an exception detection result is detected.

[0064] In one implementation, the system in the embodiment of the present invention further includes: The task data storage module is used to compress the task metadata and task parameters in the Shell script task to obtain compressed data before the target Shell script task is executed, perform block processing on the compressed data using distributed storage, and store the task data blocks after the block processing.

[0065] In one implementation, the system in the embodiment of the present invention further includes: The network synchronization module is used to obtain the priority and execution status of each Shell script task to be scheduled for execution, adopt a batch processing strategy, gradually report the execution data of each Shell script task to the central node, and synchronize data through a handshake protocol with the central node.

[0066] The system of the present invention is compatible with existing task schedulers and can be seamlessly integrated into existing systems, thereby reducing deployment difficulty and technical costs.

[0067] Embodiment 3: Based on the same inventive concept, the present invention also provides a computer device, like Figure 6 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0068] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of a Shell script task scheduling method in a network-free state in the above-mentioned embodiment.

[0069] Embodiment 4: Based on the same inventive concept, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of a Shell script task scheduling method in a network-free state in the above embodiment.

[0070] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A method for scheduling Shell script tasks without a network, characterized in that: The method comprises: When in a no-network state, according to the priority of each Shell script task to be scheduled for execution, each Shell script task is sorted according to the priority to form a task priority queue; If the preset triggering condition is met, the target Shell script task with the highest priority is extracted from the task priority queue, and the target Shell script task is submitted to the execution queue; Executing the target Shell script task in the execution queue in a no-network state; An exception detection algorithm is used to detect execution parameters during the execution of the target Shell script task; if an exception is detected, an exception handling strategy is used to perform exception handling on the target Shell script task.

2. The method according to claim 1, characterized in that According to the priority of each Shell script task to be scheduled for execution, each Shell script task is sorted according to the priority to form a task priority queue, including: Obtain the dependency of each Shell script task to be scheduled and executed, and determine the dependency relationship between every two Shell script tasks; According to the dependency relationship between each two Shell script tasks, each Shell script task is grouped to form a plurality of task sub-queues; Draw a directed graph for the Shell script tasks in each task subqueue; Performing topological sorting according to the directed graph to determine the execution order of the Shell script tasks in each task subqueue; Based on each task sub-queue, determining the highest priority in the execution order of the Shell script tasks in the task sub-queue as the priority of the task sub-queue; Sort multiple task subqueues by priority to form a task priority queue.

3. The method according to claim 1, characterized in that The abnormality detection algorithm is used to detect the execution parameters in the process of executing the target Shell script task, including: Obtaining the execution log of the target Shell script task and the data of system resource occupation, preprocessing the data, and extracting key features; Performing anomaly detection on the key features using an anomaly detection algorithm; If an exception is detected, the exception information is recorded and the state of the target Shell script task is marked as abnormal.

4. The method according to claim 3, characterized in that The adopting the exception handling strategy to perform exception handling on the target Shell script task includes: Acquire the exception type and task importance attributes of the target Shell script task in the exception information; Determine whether to retry executing the target Shell script task according to the exception type and task importance; If yes, the task retry mechanism is triggered to re-execute the target Shell script task and update the status information in the execution log; If not, the target Shell script task is skipped, and the skipping reason and status information are recorded in the execution log.

5. The method according to claim 4, characterized in that Before executing the target Shell script task in the execution queue in a non-network state, the method further includes: Obtaining task metadata and task parameters of the target Shell script task; Performing data compression processing on the task metadata and the task parameters to obtain compressed data; The compressed data is processed into blocks using distributed storage to obtain task data blocks.

6. The method according to claim 5, characterized in that After the task retry mechanism is triggered, it also includes: The compressed data is read from the task data block, and the target Shell script task is restored to a waiting state.

7. The method according to claim 1, characterized in that After executing the target Shell script task in the execution queue in a non-network state, the method further includes: When the network connection is restored, the priority and execution status of each Shell script task to be scheduled for execution are obtained; A batch processing strategy is adopted according to the priority and execution status of each Shell script task, and the execution data of each Shell script task is gradually reported to the central node, and data synchronization is performed through a handshake protocol with the central node.

8. A Shell script task scheduling system in a non-network state, characterized in that: include; The task queue module is used to sort each Shell script task according to the priority of each Shell script task to be scheduled for execution, so as to form a task priority queue; The task trigger module is used to submit the target Shell script task that meets the trigger condition to the execution queue; A task execution module, used for executing the target Shell script task in the execution queue in a non-network state; An anomaly detection module, used for performing anomaly detection on execution parameters during execution of the target Shell script task; The exception handling module is used to perform exception handling on the target Shell script task for which an exception detection result is detected.

9. The system according to claim 8, characterized in that The system further comprises: The task data storage module is used to compress the task metadata and task parameters in the Shell script task to obtain compressed data before the target Shell script task is executed, perform block processing on the compressed data using distributed storage, and store the task data blocks after the block processing.

10. The system according to claim 8, characterized in that The system further comprises: The network synchronization module is used to obtain the priority and execution status of each Shell script task to be scheduled for execution, adopt a batch processing strategy, gradually report the execution data of each Shell script task to the central node, and synchronize data through a handshake protocol with the central node.

Images