A task processing method, computer program product, device and storage medium

Abstract

The present application discloses a task processing method, a computer program product, a device and a storage medium, which are applied to the field of server technology. The main registration center component uses one of the microservices with the same function running on all servers in a server cluster in normal working state as the current main microservice of the function, and the remaining microservices as the current slave microservices of the function; any server in normal working state has at least one main microservice; the server task is dispatched to the main microservice through the main registration center component; the main first database component allows any microservice to read and only allows the main microservice to write; the slave first database component allows any microservice to read. Applying the solution of the present application, the master-slave architecture and the multi-active architecture are merged and rectified, which ensures the reliability of the server cluster while effectively improving resource utilization and is less likely to cause data inconsistency and data conflict.

Description

Technical Field

[0001] The present invention relates to the field of server technology, and in particular to a task processing method, computer program product, device and storage medium. Background Art

[0002] With the rapid development of internet technology and the continuous expansion of enterprise business, the requirements for system stability and reliability are becoming increasingly stringent. Currently, some scenarios employ a high-availability deployment solution with active-standby redundancy. This solution involves a primary server providing services while a standby server synchronizes data. In the event of a primary server failure, services can be switched to the standby server. In this solution, the standby server remains in standby mode. While this ensures service reliability, it does not run any business on the standby server, resulting in higher deployment costs and lower resource utilization.

[0003] In addition to this active-standby redundant deployment solution, some solutions utilize a multi-active architecture, where multiple servers operate simultaneously, effectively improving resource utilization. However, this architecture requires real-time data synchronization between servers. Since every server is actively working, synchronization is often complex, requiring numerous read / write locks to ensure data consistency. This, coupled with high real-time requirements, can easily lead to data inconsistencies and conflicts. Furthermore, in practice, the operational complexity of a multi-active architecture is far greater than that of an active-standby redundant architecture.

[0004] In summary, how to effectively ensure the reliability of server clusters, improve resource utilization, and prevent data inconsistencies and data conflicts is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0005] The purpose of the present invention is to provide a task processing method, computer program product, device and storage medium to effectively ensure the reliability of a server cluster, improve resource utilization, and ensure that data inconsistencies and data conflicts are unlikely to occur.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a task processing method, which is applied to any server in a server cluster, comprising:

[0008] Determine whether the registry component running on itself is the main registry component;

[0009] If not, determine that the registry center component running on itself is a slave registry center component, and synchronize data with the master registry center component;

[0010] If yes, then for the microservices with the same function running on all the servers in the server cluster in normal working state, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function; wherein, any server in normal working state has at least one master microservice;

[0011] dispatching the received server task to a main microservice for processing the server task through the main registration center component, so that when the main microservice receives the server task for writing to the first database, the write operation to the first database is performed based on the server task;

[0012] Among them, the first database is set in each server, and at the same time only one first database serves as the main first database to allow read and write operations of the main microservice and allow read operations of the slave microservice, and the remaining first databases serve as slave first databases to allow read operations of the slave microservice.

[0013] On the other hand, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0014] The main registration center component dispatches the received server tasks corresponding to the data collection and reporting functions to the main microservice in each collection microservice for processing the server tasks;

[0015] The collection microservice is a microservice running on the collection layer of the server.

[0016] On the other hand, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0017] The main registration center component dispatches the received server tasks corresponding to the authentication function to the main microservices in each platform system microservice for processing the server tasks;

[0018] The main registration center component dispatches the received server tasks corresponding to business scheduling, task timing, and patrol functions to the main microservice in each task microservice for processing the server tasks;

[0019] The main registration center component dispatches the received server tasks corresponding to the gateway functions to the main microservices in each gateway microservice for processing the server tasks;

[0020] Among them, the platform system microservice, the task microservice, and the gateway microservice are all microservices belonging to the data processing layer of the server.

[0021] On the other hand, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0022] Dispatching the received server tasks corresponding to the asset processing functions to the main microservices in each asset microservice for processing the server tasks through the main registration center component;

[0023] Dispatching the received server tasks corresponding to the device control functions to the master microservices in each control microservice for processing the server tasks through the master registration center component;

[0024] The main registration center component dispatches the received server tasks corresponding to the device initialization configuration and image mounting functions to the main microservice in each startup microservice for processing the server tasks;

[0025] Dispatching the received server tasks corresponding to asset alarm and performance monitoring functions to the master microservice in each monitoring microservice for processing the server tasks through the master registration center component;

[0026] The main registration center component dispatches the received server task corresponding to the logging function to the main microservice in each startup microservice for processing the server task;

[0027] The main registration center component dispatches the received server tasks corresponding to the intranet interface function provided to the outside world to the main microservices of each northbound microservice for processing the server tasks;

[0028] The main registration center component dispatches the received server tasks corresponding to the external network interface function to the main microservices of each external network microservice for processing the server tasks;

[0029] Among them, the asset microservice, the startup microservice, the monitoring microservice, the log microservice, the northbound microservice, and the external network microservice are all microservices running on the business processing layer of the server.

[0030] On the other hand, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0031] The main registration center component dispatches the received server tasks corresponding to the functions provided by the backend data interface to the main microservice in each web backend microservice for processing the server tasks;

[0032] The web page backend microservice is a microservice running on the interaction layer of the server.

[0033] On the other hand, it also includes:

[0034] The server's page display is performed through the interaction between the front-end service component running on its own and the web page back-end microservice.

[0035] On the other hand, a single microservice only occupies one process in the operating system.

[0036] On the other hand, it also includes:

[0037] Determine whether the address allocation component running on itself is the primary address allocation component;

[0038] If yes, allocate addresses to each microservice, each registry center component, and each first database in the server cluster, and allocate virtual addresses to each current main microservice, the main registry center component, and the main first database;

[0039] If not, the address allocation component running on itself is determined to be a slave address allocation component, and data synchronization is performed with the master address allocation component.

[0040] On the other hand, it also includes:

[0041] Based on the mutual communication between the first database running on itself and the first databases running on other servers in the server cluster, a first database is selected from the currently surviving first databases based on the load balancing principle as the current main first database.

[0042] On the other hand, it also includes:

[0043] Dispatching the received server task to a main microservice for processing the server task through the main registration center component, so that when the main microservice receives the server task for writing to the second database, the write operation to the second database is performed based on the server task;

[0044] Wherein, each server is provided with the second database, and at the same time, only one second database is used as the master second database to allow read and write operations of the master microservice and read operations of the slave microservice, and the remaining second databases are used as slave second databases to allow read operations of the slave microservice;

[0045] Each of the first databases is a relational database for persisting data; each of the second databases is a cache database for caching data.

[0046] On the other hand, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0047] Dispatching the received server task to a main microservice for processing the server task through the main registration center component, so that the main microservice executes the server task based on a message queue deployed in the server;

[0048] Among them, the message queues deployed in each of the servers are message queues that adopt a cluster multi-write mode, so that data is shared internally in the message queues in each of the servers, and the message queue data in each of the servers is consistent, and when any message in the message queue is consumed, only a single server consumes the message.

[0049] On the other hand, for the microservices with the same function running in all the servers in normal working state, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function, including:

[0050] When any master microservice is detected to be abnormal, and / or the timing reaches a preset period, a master-slave update of the microservice is triggered. Each time a master-slave update of a microservice is triggered, according to the load balancing principle, for the microservices with the same function running on all the servers in normal working conditions, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function;

[0051] Whenever a master-slave update of a microservice is triggered, the timing duration is reset to zero and restarted after being reset to zero.

[0052] In a second aspect, the present invention provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the task processing method as described above are implemented.

[0053] In a third aspect, the present invention provides a task processing device, comprising:

[0054] memory for storing computer programs;

[0055] A processor is used to execute the computer program to implement the steps of the task processing method as described above.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the task processing method as described above are implemented.

[0057] By applying the technical solution provided by the embodiment of the present invention, it is possible to merge and rectify the active-standby architecture and the multi-active architecture, thereby achieving the effect of ensuring the reliability of the server cluster and improving resource utilization.

[0058] Specifically, in the present application, each server is composed of multiple microservices, and server tasks are processed by these microservices. Server tasks are dispatched by the registration center component. Specifically, when a main registration center component itself is the main registration center component, it can perform master-slave management of microservices of various functions and dispatch server tasks. When it is a slave registration center component, it needs to synchronize registration center information with the main registration center component. When performing master-slave management of microservices, for the microservices with the same function in all servers in the server cluster under normal working conditions, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function. For a certain server task, it will be dispatched to the master microservice used to process the server task. Moreover, any server has at least one master microservice under normal working conditions. That is to say, in the present application, each server will work, and specifically, the current master microservice in the server will be dispatched server tasks by the registration center component. Since each server will work, the present application, compared with the traditional master-slave redundant architecture, ensures the reliability of the server cluster while effectively improving resource utilization.

[0059] A first database is running in each server. When the first database itself is the main first database, only the main microservice is allowed to perform its own write operations, and any microservice is allowed to perform its own read operations. When it is a slave first database, any microservice is allowed to perform its own read operations, and data can be synchronized with the main first database. It can be seen that at the same time, there will only be one first database that is the main first database to perform data write operations, and for any server function, the only main microservice under this function is allowed to perform write operations on the first database. Therefore, in this application solution, there will be no situation where multiple microservices with the same function write to the main first database at the same time, nor will there be a situation where a main microserver writes data to multiple first databases at the same time. Therefore, this application solution is not prone to data inconsistency and data conflicts.

[0060] To sum up, this application solution merges and reorganizes the master-slave architecture and the multi-active architecture, which ensures the reliability of the server cluster while effectively improving resource utilization and making it less prone to data inconsistency and data conflicts. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0062] Figure 1 This is a flowchart of a task processing method provided in a specific embodiment of the present invention;

[0063] Figure 2 A schematic diagram of the server operation deployment structure in a specific embodiment of the present invention;

[0064] Figure 3 A schematic diagram of the internal structure of a business service system in a specific embodiment of the present invention;

[0065] Figure 4 A schematic diagram of the interaction between the business service system and various components in a single server in a specific embodiment of the present invention;

[0066] Figure 5 A schematic diagram of the structure of a server cluster provided in another specific embodiment of the present invention;

[0067] Figure 6 A schematic structural diagram of a computer-readable storage medium according to the present invention;

[0068] Figure 7 A schematic diagram of the structure of a server cluster provided in a specific embodiment of the present invention;

[0069] Figure 8 A structural diagram of a task processing device provided in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0070] The core of the present invention is to provide a task processing method, computer program product, device and storage medium, which merge and rectify the active-standby architecture and the multi-active architecture, ensuring the reliability of the server cluster while effectively improving resource utilization and making data inconsistency and data conflict less likely to occur.

[0071] To help those skilled in the art better understand the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It is apparent that the embodiments described are only a portion of the present invention, not all of the embodiments. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0072] Please refer to Figure 1 and Figure 7 , Figure 1 This is an implementation flow chart of a task processing method provided in a specific embodiment of the present invention. The method can be applied to any server in a server cluster. Figure 7 This is a structural diagram of a server cluster provided by a specific embodiment of the present invention. The server cluster may include N servers connected to each other for communication, where N is a positive integer not less than 2 and may generally be set to 3.

[0073] Step S101: Determine whether the registry component running on the system is the main registry component. If not, execute step S102; if yes, execute step S103.

[0074] In the present application, a registration center component is set up in the server, and the main registration center component is required to perform master-slave management of microservices of various functions and dispatch server tasks. Therefore, it is necessary to determine whether the registration center component running by itself is the main registration center component.

[0075] Step S102: Determine the registry center component running on itself as a slave registry center component, and synchronize data with the master registry center component;

[0076] For the slave registry component, data synchronization is required with the master registry component to ensure data consistency between the various registry components.

[0077] Step S103: For the microservices with the same function running in all servers in the server cluster in normal working state, one of the microservices is used as the master microservice of the current function, and the remaining microservices are used as slave microservices of the current function; wherein, any server in normal working state has at least one master microservice.

[0078] by Figure 7 For example, the master registry component is the registry component in server 2, and the registry components in servers 1 and 3 are slave registry components. Figure 5 In the example, the master registry component is the registry component in server 3, and the registry components in servers 1 and 2 are slave registry components. Figure 7 and Figure 5Each of these examples uses three servers to form a server cluster, a common implementation used in practical applications. In this example, N = 3. Of course, in other specific scenarios, the number of servers in a server cluster can be set and adjusted as needed. However, it should be understood that regardless of the value of N, there will only be one master registration center component at any given time, with the rest acting as slave registration center components. Similarly, for the first and second databases in this application, only one will act as the master at any given time, with the rest acting as slaves.

[0079] Regarding the registration center components installed in each server, the specific rules for determining which one serves as the primary registration center component can be set according to actual needs. For example, the various registration center components can communicate with each other directly or indirectly to determine the status of each registration center component. It is also understood that a registration center component with an abnormal working status should not be used as the primary registration center component because it is not working properly. For registration center components that are working properly, for example, one can be randomly selected as the primary registration center component. For another example, it can be set to configure that as long as a specified server (such as server 1) is in a normal working state, the registration center component in that server will be the primary registration center component. For another example, the registration center component in the server with the lowest current load can be used as the primary registration center component, which is conducive to ensuring load balancing.

[0080] The main registration center component needs to manage the master-slave relationships of each microservice. The master-slave management rule for microservices is: for microservices with the same function running on each server in normal operation, one microservice is designated as the master microservice for that function, and the remaining microservices are designated as slave microservices for that function. Furthermore, any server in normal operation must have at least one master microservice.

[0081] Still taking 3 servers as an example, for example Figure 3The asset microservice shown is the Asset microservice. The master registry component can manage the master-slave relationship of the three Asset microservices. For example, the Asset microservice on server 1 is designated as the master of the three Asset microservices, while the Asset microservices on servers 2 and 3 are designated as slaves. At this point, if a server task requires processing by the Asset microservice, the master registry component will dispatch the task to the Asset microservice on server 1. For example, at some point later, the Asset microservice on server 2 becomes the master of the three Asset microservices, while the Asset microservices on servers 1 and 3 are designated as slaves. At this point, if a server task requires processing by the Asset microservice, the master registry component will dispatch the task to the Asset microservice on server 2. As can be seen, server tasks are dispatched to the master microservice responsible for handling them.

[0082] In addition, in the present application, in order to ensure resource utilization efficiency, each server will participate in the work. Therefore, for any server, as long as the server is in normal working condition, there will be at least one main microservice in the server.

[0083] Since the load conditions of different servers are constantly changing in actual applications, in the present application, the master-slave relationship of microservices with various functions can be updated, for example, it can be updated periodically. For example, based on the periodic updates, when a master microservice is abnormal, a slave microserver with the same function can be immediately switched to the master microservice.

[0084] That is, in a specific embodiment of the present invention, for the microservices with the same function running in all servers in normal working state, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function, as described in step S103, which may specifically include:

[0085] When any master microservice is detected to be abnormal, and / or the timing reaches the preset period, a master-slave update of the microservice is triggered. Whenever a master-slave update of a microservice is triggered, according to the load balancing principle, for microservices with the same function running on all servers in normal working conditions, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function;

[0086] Among them, whenever the master-slave update of the microservice is triggered, the timing duration is reset to zero and restarted after being reset to zero.

[0087] In this implementation, it is to support the periodic update of the master-slave relationship of the microservices of each function, and when any master microservice is detected to be abnormal, the master-slave update of the microservice will be triggered immediately. And when updating the master-slave relationship of the microservice, this implementation is based on the load balancing principle. For example, if the current load of server 1 is high, when updating the master-slave relationship of the microservices of each function, more microservices in server 1 can be used as the current slave microservices, and the corresponding microservices in server 2 or 3 can be used as the master microservice to reduce the load pressure of server 1 and ensure the load balance of each server. In addition, it can be understood that when any update is performed, for any microservice of a function, the microservice that should be working normally has the opportunity to be selected as the master microservice of the function, and the microservice that has an abnormality will not be selected as the master microservice of the function.

[0088] Step S104: dispatching the received server task to the main microservice for processing the server task through the main registration center component, so that when the main microservice receives the server task for writing to the first database, the write operation to the first database is performed based on the server task;

[0089] Among them, each server is equipped with a first database, and at the same time only one first database serves as the main first database to allow read and write operations of the main microservice and read operations of the slave microservice, and the remaining first databases serve as slave first databases to allow read operations of the slave microservice.

[0090] In the present application, each server runs multiple microservices for completing server tasks. For the sake of convenience, the collection of microservices in each server is called the business service system of the server. The business service system is composed of multiple microservices and is used to complete server tasks.

[0091] When the main registration center component dispatches server tasks, it dispatches them to the main microservice in the microservices of the corresponding functions according to the different server functions involved in the server tasks. It can be understood that for the same function, this application solution will only have one main microservice for that function.

[0092] The present application solution can adopt a centralized microservice deployment approach, a variation of the microservice architecture. While maintaining service independence and modularity, it introduces a centralized management component, namely the registry component described above. The registry component can be responsible for key tasks such as service coordination, configuration management, request routing, and security. This centralized management approach makes the system more unified and controllable. Furthermore, due to the independence and modularity of microservices, despite the existence of a centralized management component, each microservice can run in its own process and interact through lightweight communication mechanisms (such as HTTP, RESTful, API, etc.).

[0093] The specific number of microservices divided in the business service system and the functions of each microservice can be set and adjusted according to actual needs, as long as the server tasks can be effectively processed. And it is understandable that since each microservice has its own independence, the more it is split, the finer the functions are divided, and when a microservice in a certain server fails, the smaller the overall impact will be. But at the same time, the more it is split, the higher the maintenance cost will be, which will increase the complexity and risk of the system to a certain extent, and will also reduce the high availability of the entire platform. In actual applications, if the server requires fewer functions, all server functions can be split into a smaller number of microservices and deployed separately. For example, in extreme cases, only one microservice can be deployed in the business service system, and the microservice will complete all the functional requirements of the server. Of course, in most cases, more than one microservice is deployed in the business service system. Later Figure 3 In the implementation method, a more appropriate microservice division method is also given.

[0094] For any server, during deployment, each microservice in the business service system can be deployed in the same operating system, reducing the complexity of deployment and the difficulty of subsequent maintenance, and realizing rapid deployment of a single node and splitting between services.

[0095] In a specific embodiment of the present invention, a single microservice only occupies one process in a one-to-one correspondence in the operating system.

[0096] This implementation takes into account that the server cluster of the present application solution, through the design of master and slave microservices, ensures that even if a microservice on a single server fails, the continued operation of the server cluster will not be affected. Therefore, to effectively achieve the separation of microservices, each microservice in the server's business service system can occupy a corresponding process in the operating system. In other words, a single microservice only occupies a one-to-one corresponding process in the operating system, which is conducive to ensuring the independence of microservices.

[0097] In practical applications, please refer to Figure 2 , is a schematic diagram of the operation deployment structure of the server in a specific implementation method. The operation process of the entire server can be layered, and then the business of each layer can be divided into various microservices according to type, so that the server functions can be well realized through each microservice. Each microservice can occupy a process of the operating system to realize the division of microservices. Therefore, in actual applications, the microservices in the solution of this application can be specifically divided into microservices belonging to the collection layer of the server, microservices belonging to the data processing layer of the server, microservices belonging to the business processing layer of the server, and microservices belonging to the interaction layer of the server.

[0098] exist Figure 2 In the implementation method, the server's operation deployment structure is divided into the collection layer, data processing layer, registration center component, business processing layer and interaction layer in sequence, and a database and message queue are also set up. This is also the deployment scheme commonly adopted by the server of this application in actual applications, which can effectively meet the functional requirements of the solution of this application. Figure 2 In practical applications, the database in the implementation scheme may be composed of one or more database components.

[0099] In a specific embodiment of the present invention, dispatching the received server task to the main microservice for processing the server task through the main registration center component may include:

[0100] The main registration center component dispatches the received server tasks corresponding to the data collection and reporting functions to the main microservices in each collection microservice used to process the server tasks;

[0101] The collection microservice is a microservice running on the collection layer of the server.

[0102] See Figure 3 , is a schematic diagram of the internal structure of a business service system in a specific implementation, which includes various microservices running in the server. Figure 3In the implementation method, the microservices belonging to the collection layer of the server specifically include a collection microservice for implementing the data collection and reporting function, namely a Collector-worker microservice, which can be used to collect and report various data. Therefore, after the main registration center component receives the server task corresponding to the data collection and reporting function, it needs to dispatch the server task to the master microservice among the Collector-worker microservices used to process the server task. For example, if the Collector-worker microservice of server 2 is the master Collector-worker microservice among the Collector-worker microservices of three servers, then the server task needs to be dispatched to the Collector-worker microservice of server 2.

[0103] In a specific embodiment of the present invention, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0104] The main registration center component dispatches the received server tasks corresponding to the authentication function to the main microservices in each platform system microservice for processing server tasks;

[0105] The main registration center component dispatches the received server tasks corresponding to business scheduling, task timing, and patrol functions to the main microservices in each task microservice used to process server tasks;

[0106] The main registration center component dispatches the received server tasks corresponding to the gateway functions to the main microservices in each gateway microservice for processing the server tasks;

[0107] The platform system microservice, task microservice, and gateway microservice are all microservices in the server's data processing layer. They can all communicate with the collection microservice in the collection layer.

[0108] In this embodiment, please refer to Figure 3The microservices belonging to the data processing layer of the server include the platform system microservices, namely the System microservices, which can be used for basic functions such as authentication. It also includes task microservices, namely the Job-schedule microservices, which are used for business scheduling and basic platform scheduled tasks, patrol tasks, etc. It also includes gateway microservices, namely the Collector-gateway microservices, which are used to ensure the routing relationship between the various services on the platform. The System microservice, the Job-schedule microservice, and the Collector-gateway microservice are all connected to the collection microservice in the collection layer, namely the Collector-worker microservice. In addition Figure 3 In the implementation method, the System microservice, Job-schedule microservice, and Collector-gateway microservice can communicate with each other.

[0109] In a specific embodiment of the present invention, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0110] The main registration center component dispatches the received server tasks corresponding to the asset processing function to the main microservice in each asset microservice used to process the server tasks;

[0111] The main registration center component dispatches the received server tasks corresponding to the device control functions to the main microservice in each control microservice for processing the server tasks;

[0112] The main registration center component dispatches the received server tasks corresponding to the device initialization configuration and image mounting functions to the main microservice in each startup microservice used to handle the server tasks;

[0113] The main registration center component dispatches the received server tasks corresponding to asset alarm and performance monitoring functions to the main microservice in each monitoring microservice used to handle server tasks;

[0114] The main registration center component dispatches the received server tasks corresponding to the logging function to the main microservice in each startup microservice used to process the server tasks;

[0115] The main registration center component dispatches the received server tasks corresponding to the intranet interface function to the main microservices of each northbound microservice for processing server tasks.

[0116] The main registration center component dispatches the received server tasks corresponding to the external network interface function to the main microservices of each external network microservice for processing the server tasks;

[0117] Among them, asset microservices, startup microservices, monitoring microservices, log microservices, northbound microservices, and external network microservices are all microservices running on the business processing layer of the server.

[0118] In this embodiment, please refer to Figure 3 The microservices belonging to the server's business processing layer include the Asset microservice, which processes asset information and provides it to the various microservices in the business processing layer. Therefore, server tasks corresponding to asset processing are assigned to the Asset microservice. The Control microservice also communicates with the Asset microservice and controls devices based on asset information. Therefore, server tasks corresponding to device control are assigned to the Control microservice. The Boot microservice, also known as the Iboot microservice, communicates with both the Asset and Control microservices and performs device initialization and image installation. The Monitor microservice, also known as the Monitor microservice, communicates with the Asset microservice and monitors asset alarms and performance, generating alarm and performance data. The Logs microservice, also known as the Logs microservice, communicates with both the Asset and Monitoring microservices and records various device logs. It also communicates with the Monitor microservice and records logs during the monitoring process. It also includes a northbound microservice, the North microservice, which communicates with the control and startup microservices, enabling the server to provide an interface to the outside world, referring to the intranet. It also includes an external microservice, the Hingeclient microservice, which communicates with the monitoring and logging microservices. Similar to the North microservice, it also enables the server to provide an interface to the outside world, referring to the extranet.

[0119] In a specific embodiment of the present invention, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0120] The main registration center component dispatches the received server tasks corresponding to the functions provided by the backend data interface to the main microservice in each web backend microservice for processing the server tasks;

[0121] Among them, the web backend microservice is a microservice running on the interaction layer of the server, which can communicate with the northbound microservice and the external network microservice, and is used to implement the backend data interface function.

[0122] In this embodiment, please refer to Figure 3The microservices belonging to the interaction layer of the server include the web backend microservices, also known as the Web-facade microservices, which can provide the backend data interface for the front-end GUI. The data of the page is displayed through the data returned by this microservice.

[0123] Furthermore, in a specific embodiment of the present invention, it may also include:

[0124] The server page is displayed through the interaction between the front-end service components running on itself and the web backend microservices.

[0125] This implementation method takes into account that each server can also include: a front-end service component that is communicated with the business service system, specifically a front-end service component that is communicated with the web back-end microservice in the business service system, thereby realizing the interaction between the front-end and back-end, and realizing the page display function of the server.

[0126] See Figure 4 , a schematic diagram of the interaction between the business service system and various components in a single server in a specific implementation. In this implementation, a front-end service component, namely the cdcnode component, is provided to communicate with the business service system. This component is related to the page display function in the web page and can implement GUI page display.

[0127] In a specific embodiment of the present invention, the master micro-server that receives the server task can perform a secondary dispatch of the server task. Specifically, for some types of server tasks, since they cannot be split any further, the master micro-server that receives the server task needs to complete the server task by itself. For some types of server tasks, when the server task is not a server task that writes to a database, and the execution of the server task requires a large workload, and the server task can be further divided, the master micro-server that receives the server task can request the remaining one or more slave micro-servers to assist itself in executing the server task. In actual applications, which server tasks need to be dispatched a second time can be set in advance.

[0128] For example, in a specific situation, Figure 3The task microservice in the Job-schedule microservice is specifically Server 1's Job-schedule microservice, which is the master microservice, and Servers 2 and 3's Job-schedule microservices are slave microservices. Therefore, when the primary registration center component needs to dispatch a server task to the Job-schedule microservice, it dispatches it specifically to Server 1's Job-schedule microservice. After Server 1's Job-schedule microservice receives the server task, for example, if the task requires polling of all specified devices, Server 1's Job-schedule microservice can dispatch the task a second time. So that the remaining one or more Job-schedule microservices serving as slave microservices can assist themselves in executing the server task. For example, after the second dispatch, the Job-schedule microservice of server 1 polls devices 1-10, the Job-schedule microservice of server 2 polls devices 11-20, and the Job-schedule microservice of server 3 polls devices 21-30.

[0129] The first database is in communication with the business service system. As described above, server tasks in this application solution are dispatched to the master microservice. Some server tasks do not involve database write operations, but some do. When a server task specifically involves writing data to the database, it should only be executed by the master microservice to avoid errors.

[0130] Specifically, for the first databases in each server, one needs to be selected as the current primary first database. Of course, there are many ways to select which first database to use as the current primary first database. For example, in one specific implementation, the following steps may be included:

[0131] Based on the mutual communication between the first database running on itself and the first databases running on other servers in the server cluster, a first database is selected from the currently surviving first databases based on the load balancing principle as the current main first database.

[0132] In this implementation, after communicating with each other, the N first databases in the server cluster can select a first database from the currently surviving first databases based on the load balancing principle as the current main first database. This implementation enables database write operations to be completed in servers with lower loads, which is conducive to ensuring write efficiency.

[0133] For the master database, only the master microservice is allowed to perform its own write operations, and any microservice is allowed to perform its own read operations. The slave database, on the other hand, allows any microservice to perform its own read operations and needs to synchronize data with the master database, thus ensuring data consistency across the entire product.

[0134] In addition, it can be understood that when an abnormality occurs in the business, whether it is an abnormality in the first database or an abnormality in the microservice, since the present application solution supports timely switching of the main microservice and the main first database, there will be no simultaneous writing in multiple places, thus avoiding dirty reads and phantom reads, and providing assistance for the smooth and accurate operation of the server cluster business.

[0135] The first database may be, for example, a relational database service component for persisting data, such as a MySQL database component. A group synchronization method may be used, such that one first database has read and write permissions and serves as the master first database, while the other two first databases have only read permissions and serve as slave first databases. The slave first databases may synchronize write operations on the master first database through binlog to achieve data synchronization.

[0136] exist Figure 5 and Figure 7 In the example, the business service system of each server is connected to the main first database in server 1, which means that the main microservices of each server can perform write operations on the current main first database.

[0137] In a specific embodiment of the present invention, it may further include:

[0138] Determine whether the address allocation component running on itself is the primary address allocation component;

[0139] If yes, allocate addresses to each microservice, each registry center component, and each first database in the server cluster, and allocate virtual addresses to each current master microservice, master registry center component, and master first database;

[0140] If not, the address allocation component running on itself is determined to be a slave address allocation component, and performs data synchronization with the master address allocation component.

[0141] In this embodiment, each server can also run an address allocation component that is in communication with the business service system, which is used to allocate addresses to each microservice, each registration center component and each first database in the server cluster when it is the main address allocation component, and allocate virtual addresses to the current main microservices, main registration center components and main first databases; when it is a slave address allocation component, it synchronizes data with the main address allocation component.

[0142] This implementation takes into account that the interactions between the aforementioned components and microservices must be achieved through IP addresses (Internet Protocol Addresses). However, in this application, there are master microservices, slave microservices, master components, and slave components. The master-slave relationship is fluid, but the configuration cannot be frequently modified. For example, if data is currently being written to the first database on server 1, the next second it may be written to the first database on server 2. This implementation takes into account that this can be achieved through virtual IP addresses.

[0143] Specifically, the master address allocation component can allocate addresses to each microservice, each registry component, and each primary database in the server cluster. Furthermore, virtual addresses are allocated to each current master microservice, master registry component, and primary primary database. Furthermore, the address allocation component can also allocate addresses to each component in the following implementations, and further allocate virtual addresses to the master component. For slave address allocation components, data synchronization with the master address allocation component is required.

[0144] Taking the first database as an example, the IP address assigned to the first database in server 1 is, for example, 10.0.0.1. Correspondingly, the IP address assigned to the first database in server 2 is, for example, 10.0.0.2, and the IP address assigned to the first database in server 3 is, for example, 10.0.0.3. At this time, a virtual IP address, such as 10.0.0.4, is needed. This virtual IP address will constantly monitor the movements of the primary first database. When the primary first database is the primary first database in server 1, the virtual IP address will be assigned to the first database in server 1. At this time, the first database in server 1 will have two IP addresses, 10.0.0.1 and 10.0.0.4. For the microservice, when it needs to write data to the first database, the microservice always accesses 10.0.0.4, and the data can be written at this time. For example, if the 10.0.0.1 node fails, the first database in server 2 becomes the primary first database, and the virtual IP will immediately switch to the first database in server 2. At this time, the first database in server 2 will have two IPs, 10.0.0.2 and 10.0.0.4. At this time, the abnormal first database in server 1 does not affect the normal operation of the first database in server 2. At this time, data writing can be achieved normally, achieving high availability of the server cluster, and data will not be wrong, and it can still run smoothly.

[0145] In a specific embodiment of the present invention, it may further include:

[0146] The main registration center component dispatches the received server task to the main microservice for processing the server task, so that when the main microservice receives the server task for writing to the second database, the write operation to the second database is performed based on the server task;

[0147] Among them, each server is provided with a second database, and at the same time only one second database is used as the main second database to allow the read and write operations of the main microservice and the read operations of the slave microservice, and the remaining second databases are used as slave second databases to allow the read operations of the slave microservice;

[0148] Each first database is a relational database for persisting data; each second database is a cache database for caching data.

[0149] This implementation takes into account that more than one database component may be set up in the server. For example, a second database is also set up in this implementation. The second database is also in the form of one master and multiple slaves. The master second database can read and write, and the slave second database is read-only. Figure 5 In the example, the business service system of each server is connected to the main second database in server 2, which means that the main microservices of each server can perform write operations on the current main second database.

[0150] For example, in one specific implementation, the first databases are all relational databases used for data persistence, while the second databases are cache databases, namely Redis components, used for caching data. These caches can be stored by various microservices, improving business processing efficiency. The Redis components can be deployed as sentinels, with one secondary database having read and write permissions as the primary secondary database, while the other two secondary databases have only read permissions as secondary secondary databases. Each secondary database can monitor the health of the other, enabling a new primary secondary database to be promptly selected if a primary secondary database experiences an anomaly.

[0151] In a specific embodiment of the present invention, the main registration center component dispatches the received server task to the main microservice for processing the server task, including:

[0152] The main registration center component dispatches the received server tasks to the main microservice for processing server tasks, so that the main microservice executes the server tasks based on the message queue deployed in the server;

[0153] Among them, the message queues deployed in each server are message queues that adopt cluster multi-write mode, so that the message queues in each server share data internally, and the message queue data in each server is consistent. When consuming any message in the message queue, only a single server consumes the message.

[0154] This implementation method takes into account that a message queue can also be set up in each server for use in the execution of server tasks. And unlike the first database and the second database mentioned above, the message queue adopts a cluster multi-write mode, so that each message in the message queue can only be successfully consumed once, that is, after a microservice sends a message, it is received by another receiving side, which is the consumption process. For example, when an Asset microservice sends a data update message, the message queue in each server will share data internally, that is, the message queue in each server will synchronize this message, so that the data in the message queue of each server is consistent. Later, for example, if a microservice in Server 3 consumes the message, other microservices cannot consume the message again.

[0155] In addition, when an exception occurs in the message queue of a server, the other two servers have previously shared data, so the message queue data in each server is consistent. Therefore, when a message is not successfully consumed due to an exception, the message can be reproduced and attempted to be consumed again on the other two servers until the message is successfully consumed.

[0156] In a specific embodiment of the present invention, please refer to Figure 5 Each server may also be provided with a third database that is in communication with the business service system; wherein the third database adopts a cluster multi-write mode.

[0157] The first and second databases in the above implementation methods both adopt a one-master-multiple-slave implementation method to effectively ensure high availability and data consistency. This implementation method further takes into account that the server can also include a third database, which adopts a cluster multi-write mode to meet the storage requirements of some types of data.

[0158] For example, the third database is specifically a time series database, which is used to store time-based data information, such as performance data. When using the cluster multi-write mode, similar to the message queue mentioned above, there is no distinction between master and slave. The microservices in each server can read and write, and maintain read-write consistency.

[0159] In a specific embodiment of the present invention, please refer to Figure 5 Each server may also include a log storage query component that communicates with the business service system; wherein the log storage query component adopts a cluster multi-write mode.

[0160] In this implementation, each server also has a log storage and query component in cluster multi-write mode, regardless of master or backup. Microservices in each server can read and write, and maintain read-write consistency. The log storage and query component can specifically use ES components and has a calling relationship with the log microservice.

[0161] In practical applications, the memory allocation strategy for microservices and components can be:

[0162] iops and hingeclient microservices (each service occupies 1 / 64 of the total);

[0163] Collector-gateway microservice, System microservice, Control microservice, Collector-worker microservice, North microservice, and logs microservice (each microservice occupies 2 / 64 of the total);

[0164] Job-schedule microservice, web-facade microservice (each microservice occupies 4 / 64 of the total);

[0165] Asset microservice and Monitor microservice (each microservice occupies 6 / 64 of the total);

[0166] MySQL component, Redis component, consul-server component, rabbitmq component, and influxdb component (occupy 22 / 64 of the total).

[0167] In addition, when the server is powered on, the startup order of microservices and components can be set. For example, in one embodiment, the startup order is as follows:

[0168] The Job-schedule microservice, Collector-gateway microservice, and System microservice are started in the first batch, with a waiting time of 5 minutes. If any of the Job-schedule microservice, Collector-gateway microservice, or System microservice times out or experiences an exception, you can exit and restart the program and be prompted to check the service status.

[0169] The second batch of startups will be started only after the first batch of startups is completed normally. The second batch of startups includes the Asset microservice and the hingeclient microservice. The waiting time for the second batch of startups can also be 5 minutes. If the Asset microservice times out or encounters an exception, you can exit and restart the program and prompt to check the service status. Otherwise, you can start the third batch of startups.

[0170] The third batch of startup includes the Monitor microservice, the Control microservice, and the Collector-worker microservice. The waiting time for the third batch of startup can also be 5 minutes. If the Monitor microservice times out or encounters an exception, you can exit and restart the program and prompt to check the service status. Otherwise, you can start the fourth batch.

[0171] The fourth batch of startup includes the Web-facade microservice, North microservice, and logs microservice. The waiting time for the fourth batch of startup can also be 5 minutes. If the Web-facade microservice times out or encounters an exception, you can exit and restart the program and prompt to check the service status. Otherwise, you can execute the subsequent startup process.

[0172] By applying the technical solution provided by the embodiment of the present invention, it is possible to merge and rectify the active-standby architecture and the multi-active architecture, thereby achieving the effect of ensuring the reliability of the server cluster and improving resource utilization.

[0173] Specifically, in the present application, each server is composed of multiple microservices, and server tasks are processed by these microservices. Server tasks are dispatched by the registration center component. Specifically, when a main registration center component itself is the main registration center component, it can perform master-slave management of microservices of various functions and dispatch server tasks. When it is a slave registration center component, it needs to synchronize registration center information with the main registration center component. When performing master-slave management of microservices, for the microservices with the same function in all servers in the server cluster under normal working conditions, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function. For a certain server task, it will be dispatched to the master microservice used to process the server task. Moreover, any server has at least one master microservice under normal working conditions. That is to say, in the present application, each server will work, and specifically, the current master microservice in the server will be dispatched server tasks by the registration center component. Since each server will work, the present application, compared with the traditional master-slave redundant architecture, ensures the reliability of the server cluster while effectively improving resource utilization.

[0174] A first database is running in each server. When the first database itself is the main first database, only the main microservice is allowed to perform its own write operations, and any microservice is allowed to perform its own read operations. When it is a slave first database, any microservice is allowed to perform its own read operations, and data can be synchronized with the main first database. It can be seen that at the same time, there will only be one first database that is the main first database to perform data write operations, and for any server function, the only main microservice under this function is allowed to perform write operations on the first database. Therefore, in this application solution, there will be no situation where multiple microservices with the same function write to the main first database at the same time, nor will there be a situation where a main microserver writes data to multiple first databases at the same time. Therefore, this application solution is not prone to data inconsistency and data conflicts.

[0175] To sum up, this application solution merges and reorganizes the master-slave architecture and the multi-active architecture, which ensures the reliability of the server cluster while effectively improving resource utilization and making it less prone to data inconsistency and data conflicts.

[0176] Corresponding to the above embodiment of the task processing method, the embodiment of the present invention further provides a task processing device, a computer-readable storage medium and a computer program product, which can be referred to in correspondence with the above.

[0177] See also Figure 8 As shown, the device may include:

[0178] Memory 801, used for storing computer programs;

[0179] The processor 802 is configured to execute a computer program to implement the steps of the task processing method in any of the above embodiments.

[0180] The computer program product includes a computer program / instruction, which implements the steps of the task processing method in any of the above embodiments when executed by a processor.

[0181] See Figure 6The computer-readable storage medium 50 stores a computer program 51. When executed by a processor, the computer program 51 implements the steps of the task processing method described in any of the above embodiments. The computer-readable storage medium 50 herein includes RAM (Random Access Memory), internal memory, ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), registers, a hard disk, a removable disk, or any other form of storage medium known in the art.

[0182] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0183] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the technical solution and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A task processing method, characterized in that: Applies to any server in the server cluster, including: Determine whether the registry component running on itself is the main registry component; If not, determine that the registry center component running on itself is a slave registry center component, and synchronize data with the master registry center component; If yes, then for the microservices with the same function running on all the servers in the server cluster in normal working state, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function; wherein, any server in normal working state has at least one master microservice; dispatching the received server task to a main microservice for processing the server task through the main registration center component, so that when the main microservice receives the server task for writing to the first database, the write operation to the first database is performed based on the server task; Wherein, each server is provided with the first database, and at the same time, only one first database serves as the master first database to allow read and write operations of the master microservice and read operations of the slave microservice, and the remaining first databases serve as slave first databases to allow read operations of the slave microservice; Also includes: Dispatching the received server task to a main microservice for processing the server task through the main registration center component, so that when the main microservice receives the server task for writing to the second database, the write operation to the second database is performed based on the server task; Wherein, each server is provided with the second database, and at the same time, only one second database is used as the master second database to allow read and write operations of the master microservice and read operations of the slave microservice, and the remaining second databases are used as slave second databases to allow read operations of the slave microservice; Each of the first databases is a relational database for persisting data; each of the second databases is a cache database for caching data.

2. The task processing method according to claim 1, characterized in that: Dispatching the received server tasks to the main microservice for processing the server tasks through the main registration center component includes: The main registration center component dispatches the received server tasks corresponding to the data collection and reporting functions to the main microservice in each collection microservice for processing the server tasks; The collection microservice is a microservice running on the collection layer of the server.

3. The task processing method according to claim 1, characterized in that: Dispatching the received server tasks to the main microservice for processing the server tasks through the main registration center component includes: The main registration center component dispatches the received server tasks corresponding to the authentication function to the main microservices in each platform system microservice for processing the server tasks; The main registration center component dispatches the received server tasks corresponding to business scheduling, task timing, and patrol functions to the main microservice in each task microservice for processing the server tasks; The main registration center component dispatches the received server tasks corresponding to the gateway functions to the main microservices in each gateway microservice for processing the server tasks; Among them, the platform system microservice, the task microservice, and the gateway microservice are all microservices belonging to the data processing layer of the server.

4. The task processing method according to claim 1, characterized in that: Dispatching the received server tasks to the main microservice for processing the server tasks through the main registration center component includes: Dispatching the received server tasks corresponding to the asset processing functions to the main microservices in each asset microservice for processing the server tasks through the main registration center component; Dispatching the received server tasks corresponding to the device control functions to the master microservices in each control microservice for processing the server tasks through the master registration center component; The main registration center component dispatches the received server tasks corresponding to the device initialization configuration and image mounting functions to the main microservice in each startup microservice for processing the server tasks; Dispatching the received server tasks corresponding to asset alarm and performance monitoring functions to the master microservice in each monitoring microservice for processing the server tasks through the master registration center component; The main registration center component dispatches the received server tasks corresponding to the logging function to the main microservice in each log microservice for processing the server tasks; The main registration center component dispatches the received server tasks corresponding to the intranet interface function provided to the outside world to the main microservices of each northbound microservice for processing the server tasks; The main registration center component dispatches the received server tasks corresponding to the external network interface function to the main microservices of each external network microservice for processing the server tasks; Among them, the asset microservice, the startup microservice, the monitoring microservice, the log microservice, the northbound microservice, and the external network microservice are all microservices running on the business processing layer of the server.

5. The task processing method according to claim 1, characterized in that: Dispatching the received server tasks to the main microservice for processing the server tasks through the main registration center component includes: The main registration center component dispatches the received server tasks corresponding to the functions provided by the backend data interface to the main microservice in each web backend microservice for processing the server tasks; The web page backend microservice is a microservice running on the interaction layer of the server.

6. The task processing method according to claim 5, characterized in that: Also includes: The server's page display is performed through the interaction between the front-end service component running on its own and the web page back-end microservice.

7. The task processing method according to claim 1, characterized in that: A single microservice only occupies one process in the operating system.

8. The task processing method according to claim 1, characterized in that: Also includes: Determine whether the address allocation component running on itself is the primary address allocation component; If yes, allocate addresses to each microservice, each registry center component, and each first database in the server cluster, and allocate virtual addresses to each current main microservice, the main registry center component, and the main first database; If not, the address allocation component running on itself is determined to be a slave address allocation component, and data synchronization is performed with the master address allocation component.

9. The task processing method according to claim 1, characterized in that: Also includes: Based on the mutual communication between the first database running on itself and the first databases running on other servers in the server cluster, a first database is selected from the currently surviving first databases based on the load balancing principle as the current main first database.

10. The task processing method according to claim 1, characterized in that: Dispatching the received server tasks to the main microservice for processing the server tasks through the main registration center component includes: Dispatching the received server task to a main microservice for processing the server task through the main registration center component, so that the main microservice executes the server task based on a message queue deployed in the server; Among them, the message queues deployed in each of the servers are message queues that adopt a cluster multi-write mode, so that data is shared internally in the message queues in each of the servers, and the message queue data in each of the servers is consistent, and when any message in the message queue is consumed, only a single server consumes the message.

11. The task processing method according to any one of claims 1 to 10, characterized in that: For the microservices with the same function running in all the servers in normal working state, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function, including: When any master microservice is detected to be abnormal, and / or the timing reaches a preset period, a master-slave update of the microservice is triggered. Each time a master-slave update of a microservice is triggered, according to the load balancing principle, for the microservices with the same function running on all the servers in normal working conditions, one of the microservices is used as the current master microservice of the function, and the remaining microservices are used as the current slave microservices of the function; Whenever a master-slave update of a microservice is triggered, the timing duration is reset to zero and restarted after being reset to zero.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the task processing method according to any one of claims 1 to 11 are implemented.

13. A task processing device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the task processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the task processing method according to any one of claims 1 to 11.

Images