Task processing method and device, computer readable storage medium and electronic device
By receiving task summaries, verifying check codes, and selecting resources, the real-time and security issues of task processing in the operation and maintenance platform are resolved, improving task processing efficiency and system stability, and making it suitable for large-scale server management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-20
- Publication Date
- 2026-03-17
AI Technical Summary
In existing technologies, the task processing of operation and maintenance platforms suffers from problems such as low real-time performance, low security, and low efficiency. In particular, the system response is slow when the amount of task information data is large, and the communication method of the agent is easily controlled by third parties.
The system receives task summaries from the target server, retrieves task information from the server cluster based on task type and unique identifier, generates a checksum using a preset algorithm for verification, selects candidate servers with low resource utilization to process the task, establishes a secure communication connection, avoids centralized data acquisition, and achieves random distribution and verification of task information.
It improves the real-time performance and security of task processing, reduces the time for acquiring task information, enhances the stability and availability of the system, avoids resource waste, and meets the needs of large-scale server management.
Smart Images

Figure CN116366654B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of financial technology or other fields, and more specifically, to a task processing method, apparatus, computer-readable storage medium, and electronic device. Background Technology
[0002] With the development of automation technology, in order to solve the problems of low efficiency, slow speed, and high error rate of manual operation, enterprises build operation and maintenance platforms to realize the batch and scheduled execution of tasks, thereby reducing operation and maintenance costs and improving automation capabilities. Such operation and maintenance platform systems can execute scripts and commands on hosts in batches, distribute files to target hosts in batches, and collect files from multiple hosts in batches, realizing various batch operation and maintenance operations on multiple hosts.
[0003] In existing technologies, operation and maintenance (O&M) platforms typically consist of a CMDB (Configuration Management Database), an O&M platform server, and managed servers. Task processing between the O&M platform server and the managed servers usually involves deploying an agent on the managed server, with communication between the server and the agent, which then executes commands. However, this task processing method has the following problems:
[0004] 1. The agent periodically queries the server for pending tasks. If a task exists, it retrieves the task information and processes it; otherwise, it continues to wait. However, this method is affected by the polling interval. If the interval is too long, the system's real-time performance is low, and the real-time task distribution cannot be guaranteed. If the interval is too short, the server will be under heavy pressure, resulting in low task processing efficiency.
[0005] 2. The agent provides an external interface, such as an API. The agent listens for HTTP responses to this interface, and if it receives a task execution request, it initiates task processing. However, this method has poor security. If the API call method is obtained by a third party, that party can arbitrarily control the managed server, which is difficult to detect, resulting in a low security level.
[0006] 3. A persistent connection is established once between the server and the agent client, and subsequent information is exchanged between the two parties through this persistent connection channel. However, when the amount of returned result data is large, it can lead to slow server response and low task processing efficiency.
[0007] There is currently no effective solution to the above problems. Summary of the Invention
[0008] The present invention provides a task processing method, apparatus, computer-readable storage medium, and electronic device to at least solve the technical problem of low task processing efficiency in the prior art.
[0009] According to one aspect of the present invention, a task processing method is provided, comprising: receiving a task summary of a task to be processed from a first server via a target server, wherein the task summary includes at least a task type, a unique identifier of the task to be processed, and a checksum of the task to be processed, wherein the first server is a server in a server cluster, and each server in the server cluster is used to assign a task to the target server; obtaining task information of the task to be processed from a second server via the target server based on the task type and the unique identifier, wherein the second server is any server in the server cluster; processing the task information via the target server based on a preset algorithm to obtain a target checksum; comparing whether the checksum and the target checksum are the same to obtain a comparison result; and processing the task to be processed based on the comparison result.
[0010] Furthermore, after receiving the task summary of the task to be processed sent by the first server through the target server, the task processing method further includes: determining multiple candidate servers from the server cluster, wherein the resource utilization rate of each candidate server is less than a preset threshold; randomly selecting a candidate server from the multiple candidate servers as a third server; and obtaining the task information of the task to be processed from the third server through the target server based on the task type and unique identifier.
[0011] Furthermore, before receiving the task summary of the task to be processed sent by the first server through the target server, the task processing method also includes: sending the IP address information of the target server to the first server through the target server to perform a registration connection operation, wherein the registration connection operation is used to establish a communication connection between the target server and the first server; detecting whether the target server and the first server are successfully connected; maintaining the communication connection between the target server and the first server when the connection between the target server and the first server is successful; and sending the IP address information of the target server to a fourth server for registration connection processing when the connection between the target server and the first server fails, wherein the fourth server is any server in the server cluster other than the first server.
[0012] Further, the check code is compared with the target check code to obtain a comparison result, including: if the check code and the target check code are the same, a first comparison result is obtained; if the check code and the target check code are different, a second comparison result is obtained.
[0013] Furthermore, processing the task to be processed based on the comparison result includes: processing the task to be processed and obtaining the processing result when the comparison result is the first comparison result; and prohibiting the processing of the task to be processed when the comparison result is the second comparison result.
[0014] Furthermore, when the comparison result is the first comparison result, the task to be processed is processed to obtain the processing result, including: detecting whether the target duration is greater than or equal to the preset duration, wherein the target duration is the processing duration of the task to be processed; when the target duration is greater than or equal to the preset duration, the processing of the task to be processed is stopped and the processing result is obtained; when the target duration is less than the preset duration, the processing of the task to be processed continues.
[0015] Furthermore, when the comparison result is the first comparison result, the task to be processed is processed. After obtaining the processing result, the task processing method also includes: sending the processing result to the second server; if the second server fails, the processing result is resent at least once through the target server until the processing result is successfully sent to one of the servers in the server cluster. Each time the target server resents the processing result, it sends the processing result to the fifth server. The fifth server is any server in the server cluster other than the second server.
[0016] Furthermore, during the process of resending the processing result at least once through the target server, the task processing method also includes: counting the number of resends corresponding to each processing result whenever the processing result is resent; and prohibiting the sending of the processing result when the number of resends is greater than or equal to a preset number.
[0017] According to another aspect of the present invention, a task processing apparatus is also provided, comprising: a receiving module, configured to receive a task summary of a task to be processed sent by a first server through a target server, wherein the task summary includes at least a task type of the task to be processed, a unique identifier of the task to be processed, and a checksum of the task to be processed, wherein the first server is a server in a server cluster, and each server in the server cluster is used to assign a task to the target server; an obtaining module, configured to obtain task information of the task to be processed from a second server through the target server based on the task type and the unique identifier, wherein the second server is any server in the server cluster; a first processing module, configured to process the task information through the target server based on a preset algorithm to obtain a target checksum; a comparison module, configured to compare whether the checksum and the target checksum are the same to obtain a comparison result; and a second processing module, configured to process the task to be processed based on the comparison result.
[0018] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described task processing method at runtime.
[0019] According to another aspect of the present invention, an electronic device is also provided, the electronic device including one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are configured to run the programs, wherein the programs are configured to execute the above-described task processing method during runtime.
[0020] In this embodiment of the invention, a task digest of a task to be processed sent by a first server is received by the target server. The task digest includes at least the task type, a unique identifier, and a checksum of the task to be processed. The first server is a server in a server cluster, and each server in the server cluster is used to assign tasks to the target server. The target server obtains the task information of the task to be processed from a second server based on the task type and unique identifier. The second server is any one of the servers in the server cluster. The target server processes the task information based on a preset algorithm to obtain a target checksum. The checksum and the target checksum are compared to obtain a comparison result. The task to be processed is then processed based on the comparison result.
[0021] In the above process, since the task summary only includes a small amount of information such as the type of task to be processed, the unique identifier of the task to be processed, and the checksum of the task to be processed, the task reception time can be reduced, thereby ensuring the real-time nature of task distribution. The second server is any server in the server cluster. By obtaining the task information of the task to be processed from the second server, it is possible to avoid obtaining the task information from a single server when the task information data volume is large, thereby reducing the task information acquisition time and achieving the technical effect of improving task processing efficiency. This solves the technical problem of low task processing efficiency in the prior art. Attached Figure Description
[0022] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:
[0023] Figure 1 This is a flowchart of an optional task processing method according to an embodiment of the present invention;
[0024] Figure 2 This is a schematic diagram of an optional operating platform system according to an embodiment of the present invention;
[0025] Figure 3 This is a schematic diagram of an optional work platform system according to an embodiment of the present invention;
[0026] Figure 4 This is a schematic diagram of an optional task processing device according to an embodiment of the present invention;
[0027] Figure 5 This is a schematic diagram of an optional electronic device according to an embodiment of the present invention. Detailed Implementation
[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0030] It should be noted that the task processing method, apparatus, computer-readable storage medium and electronic device disclosed herein can be used in the field of financial technology, or in any field other than financial technology. The application fields of the task processing method, apparatus, computer-readable storage medium and electronic device disclosed herein are not limited.
[0031] It should be noted that all relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this invention are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with the relevant user or organization. Before obtaining relevant information, it needs to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent from the aforementioned user or organization.
[0032] Example 1
[0033] According to an embodiment of the present invention, a method embodiment of a task processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0034] For ease of description, some nouns or terms used in the various embodiments are explained below:
[0035] The operation platform system is a basic operation and maintenance management system with a range of fundamental capabilities, including server command execution, script execution, file distribution, and scheduled tasks. This system uses specific technologies to package user instructions into tasks, send them to the managed servers for execution, and return the execution results. It also supports batch task distribution and task flow scheduling.
[0036] Figure 1 This is a flowchart of a task processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:
[0037] Step S102: Receive the task summary of the task to be processed sent by the first server through the target server. The task summary includes at least the task type of the task to be processed, the unique identifier of the task to be processed, and the checksum of the task to be processed.
[0038] In one alternative embodiment, the target server is Figure 2 The operating platform system shown is a managed server within a managed server cluster. In this embodiment, the target server is used as the execution subject of the task processing method in this application embodiment. Figure 2 The illustrated operating platform system includes at least an operating platform server and a managed server cluster, which comprises multiple managed servers, for example... Figure 2 The system consists of managed server 1, managed server 2, ..., managed server n, with a corresponding agent deployed on each managed server. The operational platform server includes a management server cluster, a data module, and an API server (i.e., the system's external input / output interface). The management server cluster comprises multiple management servers, and the data module includes a database and a cache, for example... Figure 2 The system consists of management server 1, management server 2, ... management server m, with a controller service deployed on each management server.
[0039] In step S102, the first server is a server in a server cluster, and each server in the server cluster is used to assign tasks to the target server. In this embodiment, the server cluster may be... Figure 2 The management server cluster in the operating platform system shown has a first server that has a communication connection with the target server.
[0040] Optionally, the task summary is a short, formatted message; the unique identifier of the task to be processed may be, but is not limited to, the task ID of the task to be processed.
[0041] It should be noted that by receiving the task summary of the task to be processed sent by the first server through the target server, it is not necessary to receive the full task information of the task to be processed. This can reduce the time for the target server to receive the task, thereby ensuring the real-time nature of task distribution.
[0042] Step S104: Obtain task information of the task to be processed from the second server based on task type and unique identifier through the target server.
[0043] In step S104, the second server is any server in the server cluster. Specifically, the target server can randomly send a request to one of the servers in the server cluster to obtain task information based on the task type and unique identifier of the task to be processed. The target server can obtain the task information of the task to be processed from any server through the request, wherein the task information is all the information of the task to be processed.
[0044] Optionally, the task information of the task to be processed shall include at least the task name, the task type, and the task parameters.
[0045] Optionally, after the target server obtains the task information of the task to be processed from any server, the second server can set the task status of the task to be processed to be received.
[0046] It should be noted that by obtaining the task information of the task to be processed from any server in the server cluster, it is possible to avoid retrieving the task information from a single server when the task information data volume is large, thereby reducing the task information retrieval time and improving the efficiency of task processing.
[0047] Step S106: The target server processes the task information based on a preset algorithm to obtain the target verification code.
[0048] Step S108: Compare whether the check code is the same as the target check code to obtain the comparison result.
[0049] Step S110: Process the task to be processed based on the comparison results.
[0050] In steps S106-S110, the preset algorithm is a pre-set digest algorithm. The target server uses this digest algorithm to process the task information and obtain the target checksum. Then, the target server can process the task to be processed by comparing the checksum with the target checksum. Specifically, if the target server determines that the checksum and the target checksum are the same, it determines that the source of the task information is trustworthy, and then the target server processes the task to be processed and obtains the processing result; if the target server determines that the checksum and the target checksum are different, it determines that the source of the task information is untrustworthy, and at this time, the target server prohibits processing the task to be processed.
[0051] It should be noted that the target server verifies the task information of the task to be processed by using a checksum, which improves the security of task processing and thus improves the security of the system.
[0052] Based on the scheme defined in steps S102 to S110 above, it can be understood that in this embodiment of the invention, the task digest of the task to be processed sent by the first server is received by the target server. The task digest includes at least the task type, unique identifier, and checksum of the task to be processed. The first server is a server in a server cluster, and each server in the server cluster is used to assign tasks to the target server. The target server obtains the task information of the task to be processed from the second server based on the task type and unique identifier. The second server is any server in the server cluster. The target server processes the task information based on a preset algorithm to obtain the target checksum. The checksum and the target checksum are compared to obtain a comparison result. The task to be processed is processed based on the comparison result.
[0053] It is noteworthy that, in the above process, since the task summary only includes a small amount of information such as the type of task to be processed, the unique identifier of the task to be processed, and the checksum of the task to be processed, the task reception time can be reduced, thereby ensuring the real-time nature of task distribution. The second server is any server in the server cluster. By obtaining the task information of the task to be processed from the second server, it is possible to avoid retrieving task information from a single server when the task information data volume is large, thereby reducing the task information retrieval time and achieving the technical effect of improving task processing efficiency. This solves the technical problem of low task processing efficiency in the prior art.
[0054] In one optional embodiment, after receiving the task summary of the task to be processed sent by the first server through the target server, the target server determines multiple candidate servers from the server cluster, wherein the resource utilization of each candidate server is less than a preset threshold; then, a candidate server is randomly selected from the multiple candidate servers as the third server; then, the target server obtains the task information of the task to be processed from the third server based on the task type and unique identifier.
[0055] In this embodiment, the target server can determine multiple candidate servers with low resource utilization (i.e., servers with large idle resources, for example, servers with resource utilization of less than 20%) from the management server cluster, and then randomly select one candidate server from the multiple candidate servers as the third server. The target server can obtain the task information of the task to be processed from the third server according to the task type and unique identifier.
[0056] It should be noted that by randomly selecting a candidate server from multiple candidate servers with low resource utilization as the third server to obtain the task information of the task to be processed, it is possible to avoid concentrating the acquisition of task information on a single server when the task information data volume is large, thereby reducing the task information acquisition time and improving the efficiency of task processing.
[0057] Optionally, in the task processing method provided in this embodiment of the invention, the target server sends its IP address information to the first server for a registration connection operation, wherein the registration connection operation is used to establish a communication connection between the target server and the first server; then, it is detected whether the target server and the first server are successfully connected; when the target server and the first server are successfully connected, the communication connection between the target server and the first server is maintained; when the target server and the first server fail to connect, the target server's IP address information is sent to a fourth server for registration connection processing, wherein the fourth server is any server in the server cluster other than the first server.
[0058] In this embodiment, the agent deployed on the target server can be configured with the IP address of the management server cluster. When the operating platform system starts, the target server can send its IP address information to any server in the management server cluster for registration and connection. When the target server establishes a communication connection with the first server in the management server cluster, the target server determines whether to reselect another server in the management server cluster for communication by detecting whether the connection between the target server and the first server is successful. If the target server successfully connects with the first server, it establishes a long-lived connection with the first server. If the connection between the target server and the first server fails, the target server sends its IP address information to other servers in the management server cluster to establish a communication connection.
[0059] Optionally, if the long connection is interrupted due to the downtime of the controller service deployed on the first server, the target server will re-register and reconnect after a preset time.
[0060] Optional, such as Figure 3 As shown, the agent deployed on the target server also includes an RPC client module and multiple task execution modules. The RPC client module has functions such as establishing long connections with the RPC server of the first server, establishing short connections, receiving task summaries, requesting detailed task information, performing task verification, scheduling task execution modules, and returning task processing results. Long connections are used for message subscription, and short connections are used to obtain task information and processing results. Multiple task execution modules are used to execute tasks to be processed. Depending on the task type, there may be multiple execution modules, including but not limited to command execution modules and file download modules. Task execution modules can be added or removed according to actual business needs.
[0061] Optional, such as Figure 3As shown, the controller service deployed on the first server also includes an IP registration queue, a task server module, and a DAL data access module. The IP registration queue records the IP address information of the target servers that establish long-term connections with the first server. The task server module is used to obtain tasks from the API server and initiate tasks. The DAL data access module is used to communicate with the database and cache. Specifically, the first server's connection data module generates and maintains its own liveness flag key after a successful connection and starts the RPC server. Through the task server module, it obtains task information from the RPC server. After the target server establishes a long-term connection with the first server, the first server maintains the target server's IP address information in the IP registration queue and synchronizes it to the cache, then updates the target server's IP address information in the database. After obtaining the task information, the task server module parses the task information, generates a task digest, sends the task digest to the target server, and sets the task status to "sent."
[0062] Optionally, the API server on the job platform is used to manage user accounts, specifically including: creating user accounts, querying users, deleting users, and updating user authentication tokens. The API server can also be used to query information about target servers. It can also be used to query the processing results of tasks based on their IDs. Furthermore, the API server can receive task information from users, including the task name, destination IP address, task type, and task parameters. Based on the destination IP address, the API server can then send the task to the first server connected to the corresponding target server.
[0063] Optional, Figure 2 The users shown can be consumers or upstream server resource management systems, such as SRE (Site Reliability Engineer), CMDB, and other systems.
[0064] It should be noted that by establishing a communication connection by having the target server actively send server information (i.e., the target server's IP address information) to the management server, the situation where the target server is sent tasks by a third party is avoided, thus improving the security of task processing.
[0065] Optionally, in the task processing method provided in this embodiment of the invention, the comparison result obtained by comparing the check code with the target check code by the target server includes: obtaining a first comparison result when the check code and the target check code are the same; and obtaining a second comparison result when the check code and the target check code are different.
[0066] In this embodiment, the target server can determine whether the source of the task information is trustworthy by comparing whether the checksum is the same as the target checksum. Specifically, if the target server determines that the checksum is the same as the target checksum, it obtains a first comparison result and determines that the source of the task information is trustworthy; if the target server determines that the checksum is different from the target checksum, it obtains a second comparison result and determines that the source of the task information is untrustworthy.
[0067] It should be noted that by comparing the checksum with the target checksum to determine whether the source of the task information is trustworthy, the security of task processing can be improved, thereby enhancing the security of the system.
[0068] Optionally, in the task processing method provided in the embodiments of the present invention, the target server processes the task to be processed based on the comparison result, including: when the comparison result is a first comparison result, processing the task to be processed and obtaining a processing result; when the comparison result is a second comparison result, prohibiting the processing of the task to be processed.
[0069] It should be noted that the target server processes the pending task when the comparison result is the first comparison result, and prohibits processing the pending task when the comparison result is the second comparison result. The ability to determine whether to process the pending task by verifying the result can improve the security of task processing, thereby improving the security of the system.
[0070] Optionally, in the task processing method provided in the embodiments of the present invention, when the comparison result is the first comparison result, the target server processes the task to be processed and obtains the processing result, including: detecting whether the target duration is greater than or equal to a preset duration, wherein the target duration is the processing duration for processing the task to be processed; when the target duration is greater than or equal to the preset duration, stopping the processing of the task to be processed and obtaining the processing result; when the target duration is less than the preset duration, continuing to process the task to be processed.
[0071] In this embodiment, when the target server is processing a task, it determines whether to stop processing the task by detecting whether the processing time of the task is greater than or equal to a preset time. If the processing time is greater than or equal to the preset time, the target server stops processing the task. If the processing time is less than the preset time, the target server stops processing the task.
[0072] It should be noted that the target server determines whether to stop processing a task by detecting whether the processing time of the task is greater than or equal to a preset time. This allows the server to forcibly interrupt the task when it times out, thus ensuring the stability of the target server.
[0073] Optionally, in the task processing method provided in the embodiments of the present invention, when the comparison result is the first comparison result, the task to be processed is processed, and after obtaining the processing result, the target server sends the processing result to the second server; if the second server fails, the target server resends the processing result at least once until the processing result is successfully sent to one of the servers in the server cluster, wherein the target server sends the processing result to the fifth server each time it resends the processing result, and the fifth server is any server in the server cluster other than the second server.
[0074] In this embodiment, after processing the task and obtaining the processing result, the target server sends the result to a second server, which can be any server in the server cluster. If the second server fails, the target server resends the processing result to any server in the server cluster other than the second server, until successful transmission. For example, if the target server sends the processing result to management server 1, and management server 1 fails, the target server sends the processing result to management server 2. If management server 2 is operating normally, the target server successfully sends the processing result to management server 2; if management server 2 fails, the target server sends the processing result to management server 3, and so on, until successful transmission.
[0075] Optionally, after successfully completing the processing of the result sent to the target server, the second server can store the processing result in the cache and database, and set the pending task to complete.
[0076] It should be noted that when the control server fails (i.e., the second server fails), the controlled server (i.e., the target server) automatically initiates a reconnection and can reconnect to other normal control servers, ensuring the overall availability of the system.
[0077] Optionally, in the task processing method provided in the embodiments of the present invention, during the process of resending the processing result at least once through the target server, the number of resends corresponding to each processing result is counted each time the processing result is resent; then, when the number of resends is greater than or equal to a preset number, the sending of the processing result is prohibited.
[0078] In this embodiment, the target server determines whether to stop sending the processing result by counting the number of retransmissions during at least one retransmission process. The target server can prohibit the sending of the processing result when the number of retransmissions is greater than or equal to a preset number.
[0079] It should be noted that determining whether to stop sending the processing result by counting the number of resends can stop sending the processing result after multiple failed attempts, reducing resource waste and thus improving the stability of the target server.
[0080] Therefore, this invention provides a highly available and high-performance job platform system adapted to scenarios involving large-scale server management. This solution reduces task reception time and ensures real-time task distribution when the amount of task information or returned processing results is large. By establishing a long connection between the management server and the managed servers, only small task summaries are transmitted, reducing task reception time. When a managed server obtains specific task information, it randomly sends requests to any management server in the management server cluster. This avoids concentrating task information retrieval on a single server when the task information volume is large, reducing retrieval time and improving task processing efficiency. By having the managed server actively send server information to the management server to establish a communication connection, the management server can understand the connection status of the managed servers, preventing tasks from being sent by third parties. Verification of task information using checksums improves task processing security, thereby enhancing system security. In this solution, the job platform server uses a read-write separation design for storage. If data exists in the cache, it is retrieved first from the higher-performance cache, improving overall system performance. In this solution, if the management server fails, the managed server automatically initiates a reconnection and can reconnect to another working management server, ensuring the overall availability of the system. The number of management servers can be increased or decreased as needed, meeting the requirements for managing a large number of servers and avoiding performance degradation caused by a large number of managed servers.
[0081] Example 2
[0082] According to an embodiment of the present invention, a task processing apparatus is provided, wherein... Figure 4 A schematic diagram of an optional task processing device according to Embodiment 1 of the present invention is shown below. Figure 4 As shown, the device includes:
[0083] The receiving module 401 is used to receive a task summary of the task to be processed sent by the first server through the target server. The task summary includes at least the task type of the task to be processed, the unique identifier of the task to be processed, and the check code of the task to be processed. The first server is a server in a server cluster, and each server in the server cluster is used to assign tasks to the target server.
[0084] The acquisition module 403 is used to obtain task information of the task to be processed from the second server based on the task type and unique identifier through the target server, wherein the second server is any server in the server cluster;
[0085] The first processing module 405 is used to process the task information through the target server based on a preset algorithm to obtain the target verification code;
[0086] Comparison module 407 is used to compare whether the check code is the same as the target check code and obtain the comparison result;
[0087] The second processing module 409 is used to process the task to be processed based on the comparison results.
[0088] It should be noted that the receiving module 401, the acquiring module 403, the first processing module 405, the comparison module 407, and the second processing module 409 mentioned above correspond to steps S102 to S110 in the above embodiments. The five modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in the above embodiment 1.
[0089] Optionally, the task processing device further includes: a first determining module, a selecting module, and a first acquiring module. The first determining module is used to determine multiple candidate servers from the server cluster after receiving the task summary of the task to be processed sent by the first server through the target server, wherein the resource utilization rate of each candidate server is less than a preset threshold; the selecting module is used to randomly select one candidate server from the multiple candidate servers as a third server; the first acquiring module is used to obtain the task information of the task to be processed from the third server through the target server based on the task type and unique identifier.
[0090] Optionally, the task processing device further includes: a first sending module, a first detection module, a connection maintenance module, and a second sending module. The first sending module is used to send the IP address information of the target server to the first server for a registration connection operation before receiving the task summary of the task to be processed from the first server via the target server. This registration connection operation is used to establish a communication connection between the target server and the first server. The first detection module is used to detect whether the target server and the first server have successfully connected. The connection maintenance module is used to maintain the communication connection between the target server and the first server when the connection is successful. The second sending module is used to send the IP address information of the target server to a fourth server for registration connection processing when the connection between the target server and the first server fails. The fourth server is any server in the server cluster other than the first server.
[0091] Optionally, the comparison module is also used to obtain a first comparison result when the check code is the same as the target check code, and a second comparison result when the check code is different from the target check code.
[0092] Optionally, the second processing module further includes a task processing unit and a prohibition processing unit. The task processing unit is used to process the task to be processed and obtain the processing result when the comparison result is the first comparison result; the prohibition processing unit is used to prohibit the processing of the task to be processed when the comparison result is the second comparison result.
[0093] Optionally, the task processing unit includes a detection unit, a stop processing unit, and a continue processing unit. The detection unit is used to detect whether the target duration is greater than or equal to a preset duration, wherein the target duration is the processing duration for processing the task to be processed; the stop processing unit is used to stop processing the task to be processed and obtain the processing result when the target duration is greater than or equal to the preset duration; the continue processing unit is used to continue processing the task to be processed when the target duration is less than the preset duration.
[0094] Optionally, the task processing device further includes a third sending module and a resending module. The third sending module is used to process the task to be processed when the comparison result is the first comparison result, and after obtaining the processing result, send the processing result to the second server. The resending module is used to resend the processing result at least once through the target server if the second server fails, until the processing result is successfully sent to one of the servers in the server cluster. Each time the target server resends the processing result, it sends the processing result to the fifth server, which is any server in the server cluster other than the second server.
[0095] Optionally, the task processing device also includes a statistics module and a transmission prohibition module. The statistics module is used to count the number of retransmissions corresponding to each processing result whenever the processing result is retransmitted at least once through the target server; the transmission prohibition module is used to prohibit the transmission of the processing result when the number of retransmissions is greater than or equal to a preset number.
[0096] Example 3
[0097] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to perform the above-described task processing method at runtime.
[0098] Example 4
[0099] According to another aspect of the present invention, an electronic device is also provided, wherein, Figure 5 This is a schematic diagram of an optional electronic device according to an embodiment of the present invention, such as... Figure 5 As shown, the electronic device includes one or more processors; and a memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to run the programs, wherein the programs are configured to execute the task processing methods described above during runtime.
[0100] like Figure 5 As shown, this application embodiment provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps:
[0101] The target server receives a task digest of the pending task sent by the first server. The task digest includes at least the task type, a unique identifier, and a checksum. The first server is a server in a server cluster, where each server is used to assign tasks to the target server. The target server then retrieves the task information of the pending task from a second server (any server in the server cluster) based on the task type and unique identifier. The target server processes the task information using a preset algorithm to obtain a target checksum. The checksum and the target checksum are compared to obtain a comparison result. The pending task is then processed based on the comparison result.
[0102] Optionally, the processor may also perform the following steps when executing the program: after receiving the task summary of the task to be processed sent by the first server through the target server, determine multiple candidate servers from the server cluster, wherein the resource utilization of each candidate server is less than a preset threshold; randomly select a candidate server from the multiple candidate servers as the third server; and obtain the task information of the task to be processed from the third server through the target server based on the task type and unique identifier.
[0103] Optionally, the processor, when executing the program, also implements the following steps: before receiving the task summary of the task to be processed sent by the first server through the target server, the target server sends its IP address information to the first server to perform a registration connection operation, wherein the registration connection operation is used to establish a communication connection between the target server and the first server; detects whether the target server and the first server have successfully connected; when the target server and the first server have successfully connected, maintains the communication connection between the target server and the first server; when the target server and the first server have failed to connect, sends the target server's IP address information to a fourth server to perform registration connection processing, wherein the fourth server is any server in the server cluster other than the first server.
[0104] Optionally, the processor may further implement the following steps when executing the program: if the check code is the same as the target check code, obtain a first comparison result; if the check code is different from the target check code, obtain a second comparison result.
[0105] Optionally, the processor may also perform the following steps when executing the program: if the comparison result is the first comparison result, process the task to be processed and obtain the processing result; if the comparison result is the second comparison result, prohibit the processing of the task to be processed.
[0106] Optionally, the processor may also perform the following steps when executing the program: detect whether the target duration is greater than or equal to the preset duration, wherein the target duration is the processing duration for processing the task to be processed; when the target duration is greater than or equal to the preset duration, stop processing the task to be processed and obtain the processing result; when the target duration is less than the preset duration, continue processing the task to be processed.
[0107] Optionally, the processor may also perform the following steps when executing the program: if the comparison result is the first comparison result, process the task to be processed, and after obtaining the processing result, send the processing result to the second server; if the second server fails, resend the processing result at least once through the target server until the processing result is successfully sent to one of the servers in the server cluster. Each time the target server resends the processing result, it sends the processing result to the fifth server, which is any server in the server cluster other than the second server.
[0108] Optionally, the processor may also perform the following steps when executing the program: during the process of resending the processing result at least once through the target server, the number of resends corresponding to each processing result is counted each time the processing result is resent; when the number of resends is greater than or equal to a preset number, the sending of the processing result is prohibited.
[0109] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0110] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.
[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0113] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0114] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0115] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A task processing method characterized by, The method comprises the following steps: receiving, by a target server, a task summary of a to-be-processed task sent by a first server, wherein the task summary comprises at least a task type of the to-be-processed task, a unique identifier of the to-be-processed task, and a check code of the to-be-processed task, the first server is a server in a server cluster, and each server in the server cluster is configured to assign tasks to the target server; obtaining, by the target server, task information of the to-be-processed task from a second server based on the task type and the unique identifier, wherein the second server is any one of the servers in the server cluster; processing, by the target server, the task information based on a preset algorithm to obtain a target check code; comparing the check code with the target check code to obtain a comparison result; and processing the to-be-processed task based on the comparison result.
2. The method of claim 1, wherein, After receiving, by a target server, a task summary of a to-be-processed task sent by a first server, the method further comprises the following steps: determining a plurality of candidate servers from the server cluster, wherein the resource utilization of each candidate server is less than a preset threshold; randomly selecting one candidate server from the plurality of candidate servers as a third server; obtaining, by the target server, task information of the to-be-processed task from the third server based on the task type and the unique identifier.
3. The method of claim 1, wherein, Before receiving, by a target server, a task summary of a to-be-processed task sent by a first server, the method further comprises the following steps: sending, by the target server, IP address information of the target server to the first server for registration connection operation, wherein the registration connection operation is configured to establish a communication connection between the target server and the first server; detecting whether the target server and the first server are successfully connected; when the target server and the first server are successfully connected, maintaining the communication connection between the target server and the first server; when the target server and the first server fail to be connected, sending the IP address information of the target server to a fourth server for registration connection processing, wherein the fourth server is any one of the servers in the server cluster except the first server.
4. The method of claim 1, wherein, Comparing the check code with the target check code to obtain a comparison result comprises the following steps: when the check code and the target check code are the same, obtaining a first comparison result; when the check code and the target check code are different, obtaining a second comparison result.
5. The method of claim 4, wherein, Processing the to-be-processed task based on the comparison result comprises the following steps: when the comparison result is the first comparison result, processing the to-be-processed task to obtain a processing result; when the comparison result is the second comparison result, prohibiting processing the to-be-processed task.
6. The method of claim 5, wherein, When the comparison result is the first comparison result, processing the to-be-processed task to obtain a processing result comprises the following steps: detecting whether a target duration is greater than or equal to a preset duration, wherein the target duration is a processing duration for processing the to-be-processed task. When the target duration is greater than or equal to the preset duration, the method further comprises: stopping processing the to-be-processed task, and obtaining the processing result; 7. The method of claim 5, wherein, When the target duration is less than the preset duration, the method further comprises: continuing to process the to-be-processed task. When the comparison result is the first comparison result, the method further comprises:
8. The method of claim 7, wherein, sending the processing result to the second server; If the second server fails, re-sending the processing result at least once through the target server until the processing result is successfully sent to one of the server cluster, wherein the target server sends the processing result to a fifth server each time the processing result is re-sent, and the fifth server is any one of the server cluster except the second server. During the process of re-sending the processing result at least once through the target server, the method further comprises:
9. A task processing apparatus characterized by comprising: counting the re-sending number corresponding to the processing result each time the processing result is re-sent; when the re-sending number is greater than or equal to a preset number, prohibiting sending the processing result. The method comprises: a receiving module configured to receive, by a target server, a task summary of a to-be-processed task sent by a first server, wherein the task summary at least includes a task type of the to-be-processed task, a unique identifier of the to-be-processed task, and a check code of the to-be-processed task, the first server is a server in a server cluster, and each server in the server cluster is configured to allocate tasks for the target server; an obtaining module configured to obtain, by the target server, task information of the to-be-processed task from a second server based on the task type and the unique identifier, wherein the second server is any one of the server cluster; a first processing module configured to process, by the target server, the task information based on a preset algorithm, and obtain a target check code; 10. A computer-readable storage medium, characterized in that, a comparison module configured to compare whether the check code and the target check code are the same, and obtain a comparison result; 11. An electronic device, comprising: a second processing module configured to process the to-be-processed task based on the comparison result. The computer readable storage medium stores a computer program, wherein the computer program is configured to execute the task processing method in any one of claims 1 to 8 when running. The electronic device comprises one or more processors; a memory configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement a program for running, wherein the program is configured to execute the task processing method in any one of claims 1 to 8 when running.
Citation Information
Patent Citations
Data processing method and device
CN112291367A
Micro-service load balancing method, device, equipment, system and storage medium
CN114884886A