A server task processing method and system based on distributed key-value storage

Abstract

This application provides a server task processing method and system based on distributed key-value storage, relating to the fields of server operation and maintenance and distributed storage technology. The method includes: an agent program integrating an etcd client software development kit initiates a Watch listener on a preset task key to the etcd server and receives pushed task data; after parsing the task data and matching it with the local server operating system, it calls the corresponding local execution module to execute the task; and the execution result is written back to the corresponding result key in etcd. This application solves the performance bottlenecks, single points of failure, complex heterogeneous server adaptation, and poor scalability issues of existing centralized scheduling by offloading heterogeneous adaptation logic to the agent program locally and utilizing etcd's Watch mechanism to build a decentralized event-driven architecture, thereby improving system reliability and operational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of server operation and maintenance and distributed storage technology, and particularly relates to a server task processing method and system based on distributed key-value storage. Background Technology

[0002] In existing large-scale server operations and maintenance, to achieve batch task processing, a common approach is to have a central server push tasks to target servers or for each server to periodically poll for tasks. In the central push mode, the central server needs to maintain the connection status of all target servers and actively distribute tasks. If the central server fails or its performance is insufficient, the entire operation and maintenance system will be paralyzed, and network configuration requirements are stringent. In the polling mode, a large number of query requests will lead to wasted system resources and task execution delays. More importantly, when dealing with server clusters composed of various heterogeneous operating systems such as Linux and Windows, existing technologies typically centrally deploy the logic for adapting to different operating systems on the central server. When the central server issues tasks, it must explicitly specify the specific instructions that match the target server's operating system. This makes the central scheduling logic highly complex. When a new operating system type is added, the central server's adaptation rules must be modified, resulting in poor system scalability.

[0003] Therefore, how to avoid the performance bottleneck and single point of failure of the central server, and achieve flexible and efficient management of heterogeneous server clusters with a unified task description, has become an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a server task processing method and system based on distributed key-value storage. By integrating the etcd client software development kit (SDK) into the agent program of each managed server, the agent program can directly interact with the etcd server, thus constructing a decentralized event-driven task processing mechanism. The adaptation logic of heterogeneous operating systems is pushed down to the local of each agent program, and the central platform only needs to issue a unified task description, realizing the complete decoupling of task distribution and task execution in the architecture.

[0005] The first aspect of this application discloses a server task processing method based on distributed key-value storage, applied to a runtime server of an agent program integrated with the etcd client software development kit, the method comprising: The etcd client software development kit initiates a Watch listening to a preset task key on the etcd server to receive task data pushed by the etcd server in response to data changes of the preset task key; wherein, the preset task key is a key pre-defined in etcd for storing task information; The received task data is parsed to obtain the task type and operating system identifier. After determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from the preset local execution module set according to the task type and the operating system type of this server to execute the task corresponding to the task data. The server task processing is completed by writing the task execution result into the corresponding result key in the etcd server using the etcd client software development kit; wherein, the result key is the key in etcd used to store the execution result corresponding to the task.

[0006] Optionally, the step of initiating a Watch request for a preset task key to the etcd server via the etcd client software development kit, in order to receive task data pushed by the etcd server in response to data changes of the preset task key, includes: A long connection is established between the etcd client software development kit and the etcd server. A Watch request for the preset task key is sent to the etcd server via the long connection; The system receives task data pushed by the etcd server when the preset task key changes data via the long connection.

[0007] Optionally, after parsing the received task data to obtain the task type and operating system identifier, and determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from a preset set of local execution modules according to the task type and the operating system type of this server, including: Parse the task data to obtain the task type and the operating system identifier; Obtain the operating system type of this server and compare it with the operating system identifier; When the operating system type of this server matches the operating system identifier, an execution module that matches both the task type and the operating system type of this server is selected from the preset execution module mapping; The selected execution module is activated to perform the task.

[0008] Optionally, selecting an execution module from a preset execution module mapping that matches both the task type and the operating system type of the server includes: Based on the operating system type of this server and the task type, the corresponding execution module identifier is queried in the preset mapping table; Obtain the call information of the corresponding execution module based on the execution module identifier; The mapping table stores the correspondence between operating system type, task type and execution module identifier.

[0009] Optionally, writing the task execution result into the corresponding result key in the etcd server using the etcd client software development kit includes: Collect the standard output, error output, task start time, task end time, and execution status of the execution module; The collected information is serialized into structured data; The structured data is written to the result key using the etcd client software development kit.

[0010] Optionally, when the task data is an upgrade instruction, after parsing the received task data to obtain the task type and operating system identifier, and determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from a preset set of local execution modules according to the task type and the operating system type of this server to execute the task corresponding to the task data, including: Parse the upgrade command to obtain the download address and integrity verification code of the new version of the agent program file; Download the new version of the agent program file from the download address, and use the integrity check code to verify the integrity of the downloaded file; After verification, a local upgrade script is generated, which executes the shutdown of the old agent program, file replacement, and startup of the new agent program. Start the local upgrade script and end the current task execution.

[0011] Optionally, the local upgrade script performs the stopping of the old agent program, file replacement, and starting of the new agent program, including: The local upgrade script terminates the currently running old version of the agent program process; Move the downloaded and verified new version of the agent program file to the path of the old version of the agent program file to complete the file replacement; Start the new version of the agent process through the operating system's service manager.

[0012] Optionally, the step of moving the downloaded and verified new version of the proxy program file to the path of the old version of the proxy program file includes: In a Linux operating system environment, the mv command is used to atomically move the new version of the agent program file to the path where the old version of the agent program file is located. In the Windows operating system environment, the MoveFileEx function is used to register the new version of the agent program file as a delayed replacement, and the replacement is completed on the next system restart.

[0013] Optionally, after starting the local upgrade script, the following may also be included: When the new version of the agent program process starts, it writes an agent program status record carrying the new version number to the etcd server through the etcd client software development kit. The agent program status record is stored in a preset agent program status key.

[0014] The second aspect of this application discloses a server task processing system based on distributed key-value storage, including an etcd server, an operation and maintenance management platform, and at least one managed server running an agent program that integrates the etcd client software development kit; The etcd server is used to store pre-agreed preset task keys and corresponding task result keys, receive Watch listening requests initiated by the agent program, and push task data to all agent programs listening to the preset task key when the data changes. The operation and maintenance management platform is used to write task data to the preset task key of the etcd server and read the task execution result from the result key of the etcd server. The agent is configured as follows: The etcd client software development kit initiates a Watch listening to a preset task key on the etcd server to receive task data pushed by the etcd server in response to data changes of the preset task key; wherein, the preset task key is a key pre-defined in etcd for storing task information; The received task data is parsed to obtain the task type and operating system identifier. After determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from the preset local execution module set according to the task type and the operating system type of this server to execute the task corresponding to the task data. The server task processing is completed by writing the task execution result into the corresponding result key in the etcd server using the etcd client software development kit; wherein the result key is the key in etcd used to store the execution result corresponding to the task.

[0015] As can be seen from the above technical solution, this method establishes a long connection between each agent program and the etcd server directly through the etcd client software development kit and listens for the pre-defined task key. When a new task needs to be distributed, the operation and maintenance platform acts only as a data writer, writing a unified task description into etcd. This writing operation does not depend on any specific central scheduling server. Based on etcd's Watch push mechanism, all agent programs with normal connections will receive task events in real time and pull task data. In this process, there is no direct control flow or connection dependency between the platform and the agent program, thus fundamentally eliminating the performance bottleneck and single point of failure risk caused by the central server. After the agent program pulls the task, it parses the task type and operating system identifier. First, it performs preliminary filtering by the operating system identifier to confirm that the task is suitable for itself. Then, it uses the task type and the local operating system type as an index to look up the pre-configured local execution module mapping table. This process fully encapsulates the handling of heterogeneous system differences within their respective agent programs. The unified task description on the central platform does not need to include any operating system-specific execution details. When adapting to a new operating system, only a corresponding local execution module needs to be developed and a new agent program containing this module needs to be deployed. The scheduling logic and data distribution format of the entire system remain unchanged. Finally, the agent program writes the structured execution results back to the result key in etcd, providing the operations and maintenance platform with a globally consistent and traceable view of task status. By combining an event-driven decentralized listening mechanism with a local heterogeneous command dual-matching execution strategy, the entire system simultaneously achieves high reliability, low latency, flexible scalability, and unified management. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a server task processing method based on distributed key-value storage in an embodiment of this application; Figure 2 This is a flowchart of S100 of the server task processing method based on distributed key-value storage in the embodiments of this application; Figure 3 This is a flowchart of S200 of the server task processing method based on distributed key-value storage in the embodiments of this application; Figure 4 This is a flowchart of S230 of the server task processing method based on distributed key-value storage in the embodiments of this application; Figure 5 This is a flowchart of S300 of the server task processing method based on distributed key-value storage in the embodiments of this application; Figure 6 This is a flowchart of S230 of the server task processing method based on distributed key-value storage in the embodiments of this application; Figure 7 This is a flowchart of S233 of the server task processing method based on distributed key-value storage in the embodiments of this application; Figure 8 This is a flowchart of S2332 of the server task processing method based on distributed key-value storage in the embodiments of this application; Figure 9 This is a structural block diagram of a server task processing system based on distributed key-value storage in an embodiment of this application. Detailed Implementation

[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not limiting, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without such specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0019] In this embodiment, the prior art is first described in detail: Taking a cluster of 1,000 servers owned by a certain enterprise as an example, 600 servers run mainstream Unix-like operating systems, 300 servers run desktop operating systems, and 100 servers run domestic operating systems. The enterprise uses a batch operation and maintenance system based on message middleware. Its workflow involves maintenance personnel creating batch tasks on a central management platform. The central platform breaks down the tasks into specific instructions for each server and then pushes these instructions to the corresponding execution programs on the servers via message middleware. After receiving the instructions, the execution programs perform the corresponding operations and send the execution results back to the central platform through another message queue. The central platform collects all results and performs statistical display. This solution has significant drawbacks in actual operation. When the number of tasks executed simultaneously exceeds 500, the load on the central platform and message middleware increases significantly, and task push delays increase. When the central platform or message middleware fails, the entire operation and maintenance system stops working. Furthermore, because commands differ between different operating systems, the central platform needs to maintain instruction conversion logic for each operating system. When a new operating system is added, the central platform's code must be modified and redeployed, a process that takes several days. Meanwhile, the upgrade of the execution program requires maintenance personnel to log in to each server one by one to perform the operation. The upgrade of all one thousand servers requires multiple maintenance personnel to spend more than a week to complete.

[0020] It should be noted that the server task processing method based on distributed key-value storage provided in this application embodiment is applied to a running server of an agent program that integrates a distributed key-value storage client software development kit, such as... Figure 1 As shown, the method includes: S100: The etcd client software development kit initiates a Watch listening to the etcd server for a preset task key, so as to receive task data pushed by the etcd server in response to data changes of the preset task key; wherein, the preset task key is a key pre-agreed in etcd for storing task information; S200: Parse the received task data to obtain the task type and operating system identifier, and after determining that the operating system type of this server matches the operating system identifier, call the corresponding execution module from the preset local execution module set according to the task type and the operating system type of this server to execute the task corresponding to the task data; S300: The task execution result is written to the result key corresponding to the task in the etcd server through the etcd client software development kit to complete the server task processing; wherein, the result key is the key in etcd corresponding to the task used to store the execution result.

[0021] In this embodiment, the distributed key-value store is a distributed database system for storing key-value pairs, characterized by high availability, consistency, and partition tolerance. The client software development kit (SDK) is a set of pre-written code libraries used to simplify communication between the client and the server. The agent program is a background program running on each managed server, responsible for executing operational tasks and interacting with the distributed key-value storage service client. The preset task key is a pre-agreed key path in the distributed key-value store used to store task information; all agent programs monitor data changes under this key path. Task data is structured data containing task-related information, including at least the task type and operating system identifier. The operating system identifier indicates which operating systems the task is applicable to; for example, it can be set to a specific operating system name or a general identifier applicable to all operating systems. The local execution module set is a set of executable programs or scripts pre-deployed on the server where the agent program resides; each execution module corresponds to a specific task type and operating system combination. The result key is the key path in the distributed key-value store corresponding to each task, used to store the execution result, and typically contains a unique identifier for the task to distinguish the execution results of different tasks.

[0022] For example, the default task key can be set to ` / operations / tasks / `. When the operations and maintenance management platform needs to issue a batch task, it will write structured data containing task information to this key path. All normally running agent programs will listen for this data change event through the distributed key-value store client software development kit and obtain the task data. After parsing the task data, the agent program obtains that the task type is software installation and the operating system identifier is a general identifier. Then, the agent program obtains the operating system type of the local server, finds that it is a Unix-like operating system, and then calls the execution module corresponding to the Unix-like operating system and software installation task type from the local execution module set. After the execution module finishes running, the agent program writes the result data, including standard output, error output, start time, end time, and execution status, to the corresponding result key path in the distributed key-value store, for example: ` / operations / results / task_12345 / server_67890 / `.

[0023] Specifically, the distributed key-value store client software development kit (SDK) provides various interfaces for communication with the server, including connection establishment, data reading and writing, and event listening. When the agent program starts, it initializes the SSD, passing in the address list and relevant configuration parameters of the distributed key-value store server, and establishes a connection with the server. Then, the agent program calls the listening interface provided by the SSD to continuously listen for preset task keys. When the data under a preset task key changes, the server proactively pushes the changed data to all agents listening for that key. This event-driven model avoids frequent query requests from the agent program to the server, reducing network traffic and system resource consumption.

[0024] Furthermore, task data is stored and transmitted using a structured data format, facilitating parsing and processing by the agent program. Structured data format is a lightweight data exchange format, characterized by its ease of reading and writing, and ease of machine parsing and generation. In addition to the task type and operating system identifier, task data can also contain other task parameters, such as the software name and version number in a software installation task, and the service name in a service startup task. These parameters are used by the execution module to complete specific task operations.

[0025] Those skilled in the art will understand that the paths for preset task keys and result keys can be set according to actual needs and are not limited to the paths in the examples above. As long as all agent programs and operation and maintenance management platforms use the same key path convention, task distribution and result feedback can be achieved. Furthermore, the structure of task data can be expanded according to actual needs, adding more fields to meet the requirements of different tasks.

[0026] It should be noted that the specific implementation process of initiating a listening operation on a preset task key to the distributed key-value storage service server through a distributed key-value storage client software development kit, in order to receive task data pushed by the etcd server in response to data changes of the preset task key, is as follows: Figure 2 As shown, S100 includes: S110: Establish a long connection with the etcd server using the etcd client software development kit; S120: Send a Watch request for the preset task key to the etcd server via the long connection; S130: Receive the task data pushed by the etcd server when the preset task key changes data via the long connection.

[0027] In this embodiment, a long-lived connection refers to a connection over which multiple data packets can be sent continuously. During the connection maintenance period, if no data packets are sent, both parties need to send link detection packets to maintain the connection. Compared to short-lived connections, long-lived connections can reduce the overhead of connection establishment and termination, thus improving communication efficiency. Communication between distributed key-value storage service clients and clients typically uses long-lived connections to support real-time event push.

[0028] For example, when initializing the client software development kit (SDK), the proxy program specifies parameters such as connection timeout and heartbeat interval. The SSD establishes one or more long-lived connections with the distributed key-value storage service based on these parameters. After establishing the connection, the proxy program calls the SSD's listener method, passing in a preset task key and listener options. Listener options can include whether to listen for subkey changes, whether to retrieve historical data, etc. Upon receiving the listener request, the distributed key-value storage service adds the client to the corresponding key's listener list. When data under that key is created, updated, or deleted, the server iterates through the listener list and pushes the change event to all clients listening for that key.

[0029] Specifically, when a network interruption occurs, the client software development kit (SDK) automatically attempts to reconnect to the distributed key-value storage service. Upon successful reconnection, the client re-initiates listening requests for the preset task key and resumes receiving events from where it left off. This ensures that the agent does not lose task data even with network instability. Furthermore, the distributed key-value storage service typically employs a cluster deployment; if one node fails, the client automatically switches to another healthy node to continue communication and listening. This highly available architecture guarantees the reliability of the entire system.

[0030] Furthermore, monitoring requests can specify the scope of the monitoring, such as monitoring only changes to a specific subkey or all subkeys under a given key path. In batch operations and maintenance scenarios, different types of tasks are typically stored under different subkeys; for example, software installation tasks might be stored in ` / operations / tasks / install / `, and service startup tasks in ` / operations / tasks / start / `. The agent can then monitor only the subkeys of specific task types, reducing unnecessary event pushes.

[0031] This application is not limited to this. In some embodiments, the agent program can simultaneously monitor multiple preset task keys to handle different types of tasks. For example, the agent program can simultaneously monitor task keys and upgrade keys. When a change in a task key is detected, the agent program executes an operation and maintenance task. When a change in an upgrade key is detected, the agent program executes its own upgrade operation.

[0032] It should be noted that the specific implementation process of parsing the received task data, obtaining the task type and operating system identifier, and determining that the operating system type of this server matches the operating system identifier, and then calling the corresponding execution module from the preset local execution module set according to the task type and the operating system type of this server, is as follows: Figure 3 As shown, S200 includes: S210: Parse the task data to obtain the task type and the operating system identifier; S220: Obtain the operating system type of this server and compare it with the operating system identifier; S230: When the operating system type of this server matches the operating system identifier, select an execution module from the preset execution module mapping that matches both the task type and the operating system type of this server; S240: Start the selected execution module to perform the task.

[0033] In this embodiment, the task type indicates the specific operation content of the task, such as software installation, service startup, configuration update, file transfer, etc. The operating system identifier indicates which operating system the task applies to, such as the name of a Unix-like operating system, a desktop operating system, a domestically developed operating system, or a general identifier applicable to all operating systems. After parsing the task data, the agent program first checks whether the operating system type of the server matches the operating system identifier in the task data. If they match, the task continues to be executed; otherwise, the task is ignored.

[0034] For example, if the task data contains an operating system identifier indicating a Unix-like operating system, the agent program, after obtaining the server's operating system type and finding it to be a Unix-like operating system, determines that a match has been found. Then, based on the task type (software installation) and the server's operating system type (Unix-like operating system), the agent program searches for the corresponding execution module from the local execution module set. This set contains execution modules for different operating systems and task types, such as software installation modules for Unix-like operating systems, software installation modules for desktop-like operating systems, and service startup modules for Unix-like operating systems. Once the agent program finds the corresponding execution module, it starts that module and passes the other parameters from the task data to the execution module.

[0035] Specifically, the proxy program can obtain the operating system type of the server by calling the application programming interfaces (APIs) provided by the operating system. Different operating systems provide different interfaces to obtain system information, and the proxy program will call the corresponding interface to obtain accurate operating system information based on the type of operating system it is running on. For example, in a Unix-like operating system, the name and version number of the operating system can be obtained by reading specific system files; in a desktop operating system, operating system information can be obtained by calling functions provided by the system.

[0036] Furthermore, the local execution module set is pre-deployed on each server, and the execution module set deployed on servers operating different operating systems may differ. For example, servers running Unix-like operating systems deploy script-based execution modules, while servers running desktop operating systems deploy command-line script-based execution modules. Execution modules can take the form of executable programs, script files, dynamic link libraries, etc., as long as they can be called by the agent program to execute the corresponding tasks.

[0037] Those skilled in the art will understand that the matching rules for the operating system identifier can be set according to actual needs. For example, it can be set to exact matching, meaning that a match is only made when the operating system type of the server is exactly the same as the operating system identifier; or it can be set to fuzzy matching, meaning that a match is made as long as the operating system type of the server belongs to the category indicated by the operating system identifier. For example, if the operating system identifier is set to Unix-like operating system, then all operating systems based on Unix-like kernels will be matched successfully.

[0038] It should be noted that the specific implementation process of selecting an execution module from the preset execution module mapping that matches both the task type and the operating system type of this server is as follows: Figure 4 As shown, S230 includes: S2301: Based on the operating system type of this server and the task type, query the corresponding execution module identifier in the preset mapping table; S2302: Obtain the call information of the corresponding execution module based on the execution module identifier; The mapping table stores the correspondence between operating system type, task type and execution module identifier.

[0039] In this embodiment, the execution module mapping is a data structure that stores the correspondence between operating system type, task type, and execution module identifier. The execution module identifier uniquely identifies an execution module and can be its filename, path, unique identifier, etc. The invocation information includes the execution module's startup method, parameter passing method, etc., to ensure the agent program correctly invokes the execution module.

[0040] For example, the execution module mapping can be a two-dimensional table, where rows represent operating system types, columns represent task types, and the values ​​in the cells are the corresponding execution module identifier and invocation information. For instance, in the cells of the Unix-like operating system row and the software installation task column, the stored execution module identifier is `install\[_software.sh](_software.sh)`, and the invocation information is to execute using a shell interpreter, passing the software name and version number as parameters. After obtaining the operating system type and task type of the local server, the agent program looks up the corresponding cell in this two-dimensional table to obtain the execution module identifier and invocation information. Then, based on the invocation information, it uses the appropriate interpreter or startup method to execute the execution module.

[0041] Specifically, execution module mappings can be stored in the agent's configuration file or in a local database. The agent loads these mappings into memory upon startup for quick lookup when needed. When adding or modifying execution modules, only the mappings and corresponding module files need to be updated; the agent's code remains unchanged. This design provides excellent scalability; supporting new task types or operating systems requires only the addition of the corresponding execution modules and mappings.

[0042] Furthermore, execution module mappings can support inheritance and overriding mechanisms. For example, a generic execution module mapping can be defined, and then specific mappings can be defined for specific operating systems or task types. When searching for an execution module, the specific mapping is searched first; if not found, the generic mapping is used. This mechanism can reduce redundant mapping definitions and improve mapping maintenance efficiency.

[0043] This application is not limited to this. In some embodiments, the execution module mapping can be a hash table, where the key is a composite key consisting of the operating system type and task type, and the value is the execution module identifier and invocation information. This hash table structure can provide faster lookup speed and is suitable for scenarios with a large number of execution modules.

[0044] It should be noted that the specific implementation process of writing the task execution result into the distributed key-value storage service client using the distributed key-value storage client software development kit is as follows: Figure 5 As shown, S300 includes: S310: Collect the standard output, error output, task start time, task end time, and execution status of the execution module; S320: Serialize the collected information into structured data; S330: Write the structured data into the result key using the etcd client software development kit.

[0045] In this embodiment, the task execution result includes all relevant information generated during the execution module's operation, enabling maintenance personnel to understand the task's execution status. Standard output contains normal information output by the execution module during operation, while error output contains error information. The task start and end times are used to calculate the task's execution duration, and the execution status indicates whether the task was successfully completed; the execution status can be success, failure, or timeout.

[0046] For example, during execution, the execution module outputs normal information to the standard output stream and error information to the error output stream. The agent program captures and records the output of these two streams. Simultaneously, the agent program records the task start time when the execution module starts and the task end time when the execution module ends. Then, the agent program determines the execution status based on the execution module's exit code: if the exit code is 0, the execution status is successful; if the exit code is not 0, the execution status is failed; if the execution module does not end within the specified time, the execution status is timed out. The agent program organizes this information into structured data and then calls the write method of the distributed key-value store client software development kit to write the structured data into the corresponding result key.

[0047] Specifically, structured data can contain multiple fields, each corresponding to a type of result information. For example, it might include a `stdout` field to store standard output, a `stderr` field to store error output, a `start_time` field to store the task start time, a `end_time` field to store the task end time, and a `status` field to store the execution status. These fields can be expanded according to actual needs; for example, an `exit_code` field could be added to store the exit code of the executing module, and a `hostname` field could be added to store the hostname of the storage server.

[0048] In addition, the result key typically contains a unique identifier for the task and a unique identifier for the server, so that the operations and maintenance management platform can distinguish the execution results of different tasks and different servers. For example, the path of the result key could be / operations / results / <task_id> / <server_id> / ,in<task_id> It is the unique identifier of the task.<server_id> It is a unique identifier for the server. The operation and maintenance management platform can retrieve the execution results of a task across all servers by traversing all subkeys under a given task identifier, and then perform statistics and display.

[0049] Those skilled in the art will understand that the method for writing task execution results can be configured according to actual needs. For example, an overwrite method can be used, where each write overwrites the previous result; or an append-only method can be used, where each write adds new content to the existing result. In batch operation and maintenance scenarios, the overwrite method is usually used because each task is only executed once on each server, and only the final execution result needs to be saved.

[0050] It should be noted that, as Figure 6 As shown, when the task data is an upgrade instruction, after parsing the received task data to obtain the task type and operating system identifier, and determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from a preset set of local execution modules according to the task type and the operating system type of this server to execute the task corresponding to the task data. Step S230 further includes: S231: Parse the upgrade instruction to obtain the download address and integrity verification code of the new version of the agent program file; S232: Download the new version of the agent program file from the download address, and use the integrity check code to perform integrity verification on the downloaded file; S233: After verification, a local upgrade script is generated, which executes the shutdown of the old agent program, file replacement, and startup of the new agent program; S234: Start the local upgrade script and end the current task execution.

[0051] In this embodiment, the upgrade command is a special type of task used to upgrade the agent program itself. The upgrade command includes the download address of the new version of the agent program file and an integrity checksum. The download address is the network address where the new version of the agent program file is located; the agent program can download the file through this address. The integrity checksum is a verification value used to verify the integrity and correctness of the downloaded file, typically calculated using a hash algorithm.

[0052] For example, operations personnel upload the compiled new version of the agent program file to an internal file server, which can be accessed by all managed servers. Then, the operations personnel create an upgrade task on the operations management platform. The task type is agent upgrade, the operating system identifier is a generic identifier, and the task parameters include the download address and integrity checksum of the new version of the agent program file. The operations management platform writes this upgrade task to a preset task key in a distributed key-value store. All agents listen for this task, parse the download address and integrity checksum, and then download the new version of the agent program file from the download address over the network. After downloading, the agent calculates the hash value of the downloaded file using the same hash algorithm and compares it with the integrity checksum in the upgrade command. If the two values ​​are the same, it means the file was downloaded completely and has not been tampered with; if they are different, it means the file was corrupted or tampered with during transmission, and the agent discards the file and records the error information.

[0053] Specifically, the proxy program can use a resume function to improve download reliability. If the network is interrupted during the download, the proxy program can resume the download from the last interrupted position after the network is restored, without having to download the entire file again. Furthermore, the proxy program can set a download timeout; if the download is not completed within the specified time, it is considered a download failure and an error message is logged.

[0054] Furthermore, the local upgrade script is a script file dynamically generated by the agent program at runtime based on the server's operating system type. The script file contains all the necessary steps for the upgrade, such as stopping the old version of the agent program process, backing up the old version files, replacing the old version files, and starting the new version of the agent program. After generating the script file, the agent program starts the script asynchronously and then terminates the current task execution. This avoids deadlock issues caused by the agent program waiting for its own process to terminate.

[0055] This application is not limited to this. In some embodiments, the upgrade instruction may also include information such as the upgrade version number and the upgrade effective time. The agent program can determine whether an upgrade is needed based on the version number. If the currently running version number is equal to or higher than the version number in the upgrade instruction, the upgrade instruction is ignored. The upgrade effective time can be used to specify that the agent program should perform the upgrade at a specific point in time to avoid performing upgrade operations during peak business periods.

[0056] It should be noted that the specific implementation process of the local upgrade script in stopping the old agent, replacing files, and starting the new agent is as follows: Figure 7 As shown, S233 includes: S2331: The local upgrade script terminates the currently running old version agent program process; S2332: Move the downloaded and verified new version of the agent program file to the path of the old version of the agent program file to complete the file replacement; S2333: Start the new version of the agent process through the operating system service manager.

[0057] In this embodiment, the operating system service manager is a component in the operating system used to manage system services, responsible for operations such as starting, stopping, restarting, and querying the status of services. Different operating systems provide different service managers, such as systemd in Unix-like operating systems and service control managers in desktop operating systems.

[0058] For example, the local upgrade script first obtains the process identifier of the currently running old version of the agent program. Then, the script sends a termination signal to the process and waits for it to exit. If the process does not exit within a specified time, the script can send a forced termination signal to forcibly terminate the process. After the process terminates, the script moves the downloaded and verified new version of the agent program file to the path where the old version of the agent program file is located, overwriting the old version file. Then, the script calls the operating system service manager's startup command to start the agent program service. The operating system service manager will start the new version of the agent program process according to the information in the service configuration file.

[0059] Specifically, when stopping the old version of the agent process, the script can use a graceful termination method, that is, first send a notification signal to allow the agent to complete the currently executing task, and then exit. This can avoid task execution interruption and ensure data consistency. If the agent does not complete the current task and exit within the specified time, the script then sends a forced termination signal.

[0060] In addition, the script can back up the old version of the agent program files before replacing the existing ones. If problems occur during the upgrade process, the system can be restored to the old version using the backup file, ensuring system availability. The backup file can be stored in the same directory as the old version file, and the filename can include the old version number and the backup time to distinguish between different backup files.

[0061] Those skilled in the art will understand that the operating system service manager's configuration file needs to be configured with an automatic restart policy, meaning that when the agent process unexpectedly exits, the service manager will automatically restart the process. This ensures the high availability of the agent program, allowing it to automatically resume operation even if it crashes. During the upgrade process, after the script terminates the old version process, the service manager will automatically start the new version process, thus achieving a smooth upgrade of the agent program.

[0062] It should be noted that the specific implementation method for moving the downloaded and verified new version of the agent program file to the path of the old version of the agent program file varies depending on the operating system. Figure 8 As shown, S2332 includes: S23321: In the Linux operating system environment, the mv command is used to atomically move the new version of the agent program file to the path where the old version of the agent program file is located; S23322: In the Windows operating system environment, the MoveFileEx function is used to register the new version of the agent program file as a delayed replacement, and the replacement is completed on the next system restart.

[0063] In this embodiment, atomic file replacement means that the file replacement operation either succeeds completely or fails completely, without any partial file replacement. Atomicity ensures that even in the event of system crashes, power outages, or other abnormal situations during the replacement process, files will not be corrupted or lost.

[0064] For example, in a Unix-like operating system environment, the move command is an atomic operation. The script first downloads the new version of the agent file to a temporary directory, and then uses the move command to move the new version file from the temporary directory to the path where the old version file is located. Because the move command is an atomic operation, even if a system crash occurs during the move, there will be no situation where the old file is deleted while the new file is not fully written. After the system recovers, either the old file still exists, or the new file has completely replaced the old file.

[0065] In desktop operating system environments, the old version of the agent program file cannot be directly overwritten because it is already running. Therefore, the script uses a system-provided function to register the new version file as a delayed replacement. This function records the paths of the new and old versions of the file in the system registry. When the system restarts, it automatically replaces the old version with the new one. This method ensures that the file replacement operation can still be completed even when the file is being used.

[0066] Furthermore, in some operating system environments, the file replacement method may differ from that of Unix-like or desktop operating systems. The agent program will select the appropriate file replacement method based on the operating system type of the server. For example, some domestically developed operating systems may provide a dedicated file replacement interface to achieve atomic file replacement operations.

[0067] This application is not limited to this. In some embodiments, the agent program can use a dual-directory approach to implement file replacement. That is, the agent program is installed in two different directories: one is the currently running directory, and the other is a backup directory. During the upgrade, the script downloads the new version file to the backup directory, then modifies the system service manager's configuration file to point the service's startup path to the backup directory. Then, the service is restarted, and the service manager starts the new version of the agent program from the backup directory. If the upgrade is successful, the roles of the currently running directory and the backup directory are swapped; if the upgrade fails, the configuration file is changed back to the original path, and restarting the service restores the old version. This method enables a more secure and reliable upgrade.

[0068] It should be noted that after starting the local upgrade script, the following operations are also required, and S234 further includes: When the new version of the agent program process starts, it writes an agent program status record carrying the new version number to the etcd server through the etcd client software development kit. The agent program status record is stored in a preset agent program status key.

[0069] In this embodiment, the agent status key is a key path in a distributed key-value store used to store agent program status information. Each agent program has a corresponding status key, used to store information such as the agent program's version number, running status, and last heartbeat time. The operation and maintenance management platform can monitor changes in the agent status key to understand the running status and version information of all agent programs in real time.

[0070] For example, after the new version of the agent program starts, it first reads its own version number. Then, the agent program calls the write method of the distributed key-value store client software development kit to write a status record containing the new version number into the corresponding agent status key. The status record can also contain information such as the agent program's startup time and running status. The operation and maintenance management platform can obtain the version numbers of all agent programs by traversing all agent status keys, and then count the number of agent programs with different versions to understand the upgrade progress. If it is found that an agent program has failed to upgrade, manual intervention can be performed.

[0071] Specifically, the agent program can periodically write heartbeat information to the agent status key to indicate that it is running normally. The heartbeat information can include the current timestamp, running status, etc. The operation and maintenance management platform can determine whether the agent program is online by checking the timestamp of the heartbeat information. If the heartbeat information of an agent program has not been updated for a certain period of time, the agent program is determined to be offline and an alarm is issued.

[0072] In addition, the agent status key can also be used to store the agent's configuration information. The operations and maintenance management platform can dynamically adjust the agent's running parameters, such as the task key path to be monitored, the path of the execution module, and the log level, by modifying the configuration information in the agent status key. After the agent detects a change in the agent status key, it will automatically load the new configuration information without restarting the agent.

[0073] Those skilled in the art will understand that the path to the agent status key can be set according to actual needs. For example, it can be set to / operations / agents / <agent_id> / status / , where<agent_id> It is a unique identifier for the agent program. The status information of each agent program is stored under its own subkey for easy management and querying.

[0074] For example, consider applying the above technical solution to the scenario mentioned earlier involving a cluster of 1,000 heterogeneous servers in a certain enterprise. The enterprise first deploys an agent program on each server, integrating a distributed key-value store client software development kit. After starting, the agent program establishes a long-lived connection with the distributed key-value store client and listens for the preset task key ` / operations / tasks / `. Simultaneously, the agent program writes its version number and running status to the corresponding agent status key ` / operations / agents / `.<agent_id> In ` / status / `, the operations and maintenance management platform reads these agent status keys to obtain the online status and version information of all agent programs.

[0075] When an operations and maintenance (O&M) engineer needs to install software on all servers in bulk, they create a task on the O&M management platform. The task type is set to software installation, the operating system identifier is set to a generic identifier, and the task parameters include the software name and version number. The O&M management platform writes the structured data of this task to the ` / operations / tasks / ` key in the distributed key-value store. Once the distributed key-value storage service detects a data change to this key, it pushes the task data to all agents listening on that key.

[0076] After receiving the task data, the agent program on each server parses out the task type and operating system identifier. The agent program obtains the operating system type of its local server, finds a match with the operating system identifier, and then continues executing the task. Next, based on the task type (software installation) and the local server's operating system type, the agent program searches for the corresponding execution module in the local execution module mapping. For example, the execution module found by the agent program for a Unix-like operating system is `install\[_software.sh](_software.sh)`, the execution module found by the agent program for a desktop operating system is `install_software.ps1`, and the execution module found by the agent program for a domestically developed operating system is the corresponding package management tool encapsulation module.

[0077] The agent program starts the corresponding execution module, passing the software name and version number as parameters. During the execution module's operation, the agent program captures standard output and error output, and records the task start and end times. After the execution module completes, the agent program determines the execution status based on the module's exit code. Then, the agent program organizes this result information into structured data and writes it to the corresponding result key / operations / results / task_12345 / in the distributed key-value store.<agent_id> / middle.

[0078] The operations and maintenance management platform retrieves task execution results from all servers by traversing all subkeys under ` / operations / results / task_12345 / `. Then, the platform displays the statistics for the number of servers that successfully executed, the number that failed, and the number that timed out, presenting these statistics to operations and maintenance personnel in a chart format. Personnel can click to view detailed execution results for each server, including standard output, error output, and execution time. For servers that failed, personnel can troubleshoot based on the error output information and then reissue the task.

[0079] When an operations and maintenance (O&M) engineer needs to upgrade all agent programs to a new version, they first upload the compiled new version agent program files to the internal file server. Then, they create an upgrade task on the O&M management platform. The task type is set to agent upgrade, the operating system identifier is set to a generic identifier, and the task parameters include the download address and integrity checksum of the new version agent program files. The O&M management platform then writes this upgrade task to the ` / operations / tasks / ` key in the distributed key-value store.

[0080] After all agents detect the upgrade task, they parse the download address and integrity checksum. Then, the agents download the new version of the agent file from the download address. Once downloaded, the agents calculate the hash value of the downloaded file and compare it with the integrity checksum. If the verification passes, the agents dynamically generate a local upgrade script based on the server's operating system type. For example, agents for Unix-like operating systems generate a shell script, while agents for desktop operating systems generate a batch script.

[0081] The local upgrade script includes the following steps: First, gracefully stop the currently running old version agent process; then, back up the old version agent file; next, atomically move the new version agent file to the path where the old version file is located; finally, start the agent service through the operating system service manager. The agent starts the local upgrade script asynchronously and then terminates the current task execution.

[0082] During script execution, the old version of the agent process is first stopped. Because the operating system's service manager has an automatic restart policy, the new version of the agent process will automatically start after the old version terminates. Once started, the new version of the agent reads its own version number and then writes the new version number into the corresponding agent status key.

[0083] The operations and maintenance management platform monitors changes in agent status keys to track the upgrade progress of all agent programs in real time. When all agent program version numbers are updated to the new version, the upgrade task is considered complete. If any agent program's version number is not updated, operations and maintenance personnel can check the task execution results for that agent program, investigate the cause of the upgrade failure, and then reissue the upgrade task.

[0084] Furthermore, the method provided in this application can also be applied to other batch operation and maintenance scenarios, such as batch configuration updates, batch service restarts, and batch file transfers. In the batch configuration update scenario, operations and maintenance personnel can create a configuration update task on the operations and maintenance management platform and write the new configuration file content into the task data. After receiving the task, the agent program writes the configuration file content to the specified local path and then restarts the corresponding service. In the batch service restart scenario, operations and maintenance personnel can create a service restart task and specify the name of the service to be restarted. After receiving the task, the agent program calls the restart command of the operating system service manager to restart the specified service.

[0085] It should be noted that the above embodiments are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0086] Those skilled in the art will understand that the technical solution of this application can be implemented through software, hardware, or a combination of both. For example, the agent program can be implemented as a standalone software program running on various operating systems; alternatively, some functions can be integrated into the kernel of the operating system to improve execution efficiency. Furthermore, the distributed key-value storage service client can employ existing open-source distributed key-value storage systems or a self-developed distributed key-value storage system.

[0087] The server task processing method based on distributed key-value storage based on the above embodiments, such as Figure 9 As shown in the figure, this application embodiment also provides a server task processing system based on distributed key-value storage, including an etcd server, an operation and maintenance management platform, and at least one managed server running an agent program that integrates the etcd client software development kit; The etcd server is used to store pre-agreed preset task keys and corresponding task result keys, receive Watch listening requests initiated by the agent program, and push task data to all agent programs listening to the preset task key when the data changes. The operation and maintenance management platform is used to write task data to the preset task key of the etcd server and read the task execution result from the result key of the etcd server. The agent is configured as follows: The etcd client software development kit initiates a Watch listening to a preset task key on the etcd server to receive task data pushed by the etcd server in response to data changes of the preset task key; wherein, the preset task key is a key pre-defined in etcd for storing task information; The received task data is parsed to obtain the task type and operating system identifier. After determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from the preset local execution module set according to the task type and the operating system type of this server to execute the task corresponding to the task data. The server task processing is completed by writing the task execution result into the corresponding result key in the etcd server using the etcd client software development kit; wherein the result key is the key in etcd used to store the execution result corresponding to the task.

[0088] This embodiment achieves highly reliable and scalable batch operation and maintenance of heterogeneous servers through a collaborative architecture of etcd server, operation and maintenance management platform and managed server.

[0089] By using the operation and maintenance management platform as only a data writing and result reading node, there is no need to maintain the connection status of a massive number of managed servers, nor is there a need to write instruction adaptation logic for different operating systems. Only a unified and abstract task description needs to be issued to complete the batch task distribution, which greatly simplifies the central scheduling logic and reduces the platform development and maintenance costs.

[0090] By deploying a highly available cluster of etcd servers and using the native Watch mechanism, real-time proactive push of task data is achieved, fundamentally eliminating the single point of failure risk and performance bottleneck of centralized scheduling architecture, while avoiding resource waste and task delays in polling mode, and ensuring the real-time and reliable distribution of tasks.

[0091] An agent program integrated with the etcd client software development kit on the managed server autonomously monitors tasks, matches operating system identifiers, and calls local execution modules to complete tasks. This completely pushes heterogeneous system adaptation logic down to the edge nodes. When a new operating system type is added, only the corresponding local execution module needs to be deployed, without modifying the central platform code. Finally, the agent program uniformly writes the structured execution results back to the etcd server, providing the operation and maintenance management platform with a globally consistent and traceable task status view, balancing system flexibility and unified control. Those skilled in the art will understand that the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. Further details are omitted here.

Claims

1. A server task processing method based on distributed key-value storage, characterized in that, The method, applied to a runtime server of an agent program integrated with the etcd client software development kit, includes: The etcd client software development kit initiates a Watch listening to a preset task key on the etcd server to receive task data pushed by the etcd server in response to data changes of the preset task key; wherein, the preset task key is a key pre-defined in etcd for storing task information; The received task data is parsed to obtain the task type and operating system identifier. After determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from the preset local execution module set according to the task type and the operating system type of this server to execute the task corresponding to the task data. The server task processing is completed by writing the task execution result into the corresponding result key in the etcd server using the etcd client software development kit; wherein, the result key is the key in etcd used to store the execution result corresponding to the task.

2. The method according to claim 1, characterized in that, The step of initiating a Watch request for a preset task key via the etcd client software development kit to receive task data pushed by the etcd server in response to data changes of the preset task key includes: A long connection is established between the etcd client software development kit and the etcd server. A Watch request for the preset task key is sent to the etcd server via the long connection; The system receives task data pushed by the etcd server when the preset task key changes data via the long connection.

3. The method according to claim 1, characterized in that, The process involves parsing the received task data to obtain the task type and operating system identifier, and determining that the operating system type of the local server matches the operating system identifier. Then, based on the task type and the operating system type of the local server, the corresponding execution module is called from a preset set of local execution modules, including: Parse the task data to obtain the task type and the operating system identifier; Obtain the operating system type of this server and compare it with the operating system identifier; When the operating system type of this server matches the operating system identifier, an execution module that matches both the task type and the operating system type of this server is selected from the preset execution module mapping; The selected execution module is activated to perform the task.

4. The method according to claim 3, characterized in that, The step of selecting an execution module from a preset execution module mapping that matches both the task type and the operating system type of the server includes: Based on the operating system type of this server and the task type, the corresponding execution module identifier is queried in the preset mapping table; Obtain the call information of the corresponding execution module based on the execution module identifier; The mapping table stores the correspondence between operating system type, task type and execution module identifier.

5. The method according to claim 1, characterized in that, The step of writing the task execution result into the corresponding result key in the etcd server using the etcd client software development kit includes: Collect the standard output, error output, task start time, task end time, and execution status of the execution module; The collected information is serialized into structured data; The structured data is written to the result key using the etcd client software development kit.

6. The method according to claim 1, characterized in that, When the task data is an upgrade instruction, the parsing process obtains the task type and operating system identifier from the received task data. After determining that the operating system type of the local server matches the operating system identifier, the corresponding execution module is called from a preset set of local execution modules according to the task type and the operating system type of the local server to execute the task corresponding to the task data, including: Parse the upgrade command to obtain the download address and integrity verification code of the new version of the agent program file; Download the new version of the agent program file from the download address, and use the integrity check code to verify the integrity of the downloaded file; After verification, a local upgrade script is generated, which executes the shutdown of the old agent program, file replacement, and startup of the new agent program. Start the local upgrade script and end the current task execution.

7. The method according to claim 6, characterized in that, The local upgrade script executes the shutdown of the old agent program, file replacement, and startup of the new agent program, including: The local upgrade script terminates the currently running old version of the agent program process; Move the downloaded and verified new version of the agent program file to the path of the old version of the agent program file to complete the file replacement; Start the new version of the agent process through the operating system's service manager.

8. The method according to claim 7, characterized in that, The step of moving the downloaded and verified new version of the proxy program file to the path of the old version of the proxy program file includes: In a Linux operating system environment, the mv command is used to atomically move the new version of the agent program file to the path where the old version of the agent program file is located. In the Windows operating system environment, the MoveFileEx function is used to register the new version of the agent program file as a delayed replacement, and the replacement is completed on the next system restart.

9. The method according to claim 6, characterized in that, After starting the local upgrade script, the following is also included: When the new version of the agent program process starts, it writes an agent program status record carrying the new version number to the etcd server through the etcd client software development kit. The agent program status record is stored in a preset agent program status key.

10. A server task processing system based on distributed key-value storage, characterized in that, This includes the etcd server, the operation and maintenance management platform, and at least one managed server running an agent program that integrates the etcd client software development kit; The etcd server is used to store pre-agreed preset task keys and corresponding task result keys, receive Watch listening requests initiated by the agent program, and push task data to all agent programs listening to the key when the preset task key changes data. The operation and maintenance management platform is used to write task data to the preset task key of the etcd server and read the task execution result from the result key of the etcd server. The agent is configured as follows: The etcd client software development kit initiates a Watch listening to a preset task key on the etcd server to receive task data pushed by the etcd server in response to data changes of the preset task key; wherein, the preset task key is a key pre-defined in etcd for storing task information; The received task data is parsed to obtain the task type and operating system identifier. After determining that the operating system type of this server matches the operating system identifier, the corresponding execution module is called from the preset local execution module set according to the task type and the operating system type of this server to execute the task corresponding to the task data. The server task processing is completed by writing the task execution result into the corresponding result key in the etcd server using the etcd client software development kit; wherein the result key is the key in etcd used to store the execution result corresponding to the task.

Images