Operation fault takeover method and device under unitized architecture

Abstract

The invention provides an operation fault takeover method under a unitized architecture. The operation fault takeover method can be applied to the technical field of artificial intelligence and distribution. The operation fault takeover method under the unitized architecture comprises the following steps: for at least one service operation accessed to a service system, dividing computing resources corresponding to the service operation into a first unit and a second unit; obtaining first health data of the first unit and second health data of the second unit in response to an initialization request of the business operation; and under the condition that the first health data does not meet the target condition and the second health data meets the target condition, loading the first data fragment of the first unit and the second data fragment of the second unit at the same time in the second unit so as to execute the business operation in the second unit. The invention further provides an operation fault takeover device under the unitized architecture, equipment, a storage medium and a program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of finance, in particular to the field of artificial intelligence and distributed technology, and more particularly to a job failure takeover method, device, equipment, medium and program product under a unit architecture. BACKGROUND

[0002] With the application of the unit architecture, the failure range of online transactions is reduced to a single unit, and some high-availability mechanisms are used to introduce traffic into a backup unit to ensure business lossless. The related batch jobs matching the same are also deployed in a unit manner, and the jobs in a single unit only process the data of the current unit. When facing unit failure, a takeover switch is usually designed, and when the unit fails, the switch is manually turned on to enable the jobs of the backup unit to take over the business of the failed unit shard. However, this approach relies on manual inspection and intervention, and the failure emergency process is long, and cannot achieve automatic takeover and automatic recovery of failures. SUMMARY

[0003] In view of the above problems, the present application provides a job failure takeover method, device, equipment, medium and program product under a unit architecture with improved high-availability capability.

[0004] According to a first aspect of the present application, a job failure takeover method under a unit architecture is provided, the method comprising: for at least one business job accessing a business system, dividing computing resources corresponding to the business job into a first unit and a second unit; in response to an initialization request of the business job, obtaining first health data of the first unit and second health data of the second unit; and in a case where the first health data does not satisfy a target condition and the second health data satisfies the target condition, simultaneously loading, in the second unit, first data shards of the first unit and second data shards of the second unit, to execute the business job in the second unit.

[0005] According to an embodiment of the present application, the method further comprises: in a case where the first health data satisfies the target condition and the second health data does not satisfy the target condition, simultaneously loading, in the first unit, the first data shards and the second data shards, to execute the business job in the first unit; or in a case where neither the first health data nor the second health data satisfies the target condition, terminating execution of the business job.

[0006] According to an embodiment of the present application, the first health data of the first unit is obtained, including: obtaining the number of live containers of the first unit; updating the abnormality count as m+1 when the number of live containers of the first unit is less than or equal to a first preset threshold, wherein m represents the current abnormality count value and m≥0; and determining that the first health data of the first unit does not satisfy the target condition when the abnormality count is greater than or equal to a second preset threshold.

[0007] According to an embodiment of the present application, the method further includes: resetting the abnormality count as an initial value when the number of live containers of the first unit is greater than the first preset threshold, and determining that the first health data of the first unit satisfies the target condition; and loading the first data fragment of the first unit for processing and loading the second data fragment of the second unit for processing.

[0008] According to an embodiment of the present application, the method further includes: sleeping for a preset sleep period after completing the detection of the current first health data; and triggering the live detection of the first health data in response to the end of the sleep period.

[0009] According to an embodiment of the present application, the method further includes: storing the first health data of the first unit and the second health data of the second unit into a storage architecture deployed in a plurality of data centers, wherein each data center independently stores full health data, and data synchronization is achieved through a semi-synchronous replication mechanism in a non-automatic degradation mode.

[0010] According to an embodiment of the present application, the loading of the first data fragment of the first unit and the second data fragment of the second unit in the second unit simultaneously to execute the business job in the second unit includes: promoting the backup library corresponding to the first unit to a new master library of the second unit, taking over the first data fragment of the original master library of the first unit; and configuring batch job nodes of the second unit to connect the original master library of the second unit to process the second data fragment of the second unit, and to connect the new master library after promotion to process the first data fragment of the first unit.

[0011] The second aspect of the present application provides a job fault takeover device under a unit architecture, comprising: a division module configured to divide, for at least one service job of an access service system, a computing resource corresponding to the service job into a first unit and a second unit; an acquisition module configured to acquire first health data of the first unit and second health data of the second unit in response to an initialization request of the service job; and a job module configured to, in a case where the first health data does not satisfy a target condition and the second health data satisfies the target condition, simultaneously load, in the second unit, a first data shard of the first unit and a second data shard of the second unit, so as to execute the service job in the second unit.

[0012] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0013] The fourth aspect of the present application further provides a computer-readable storage medium having stored thereon a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method.

[0014] The fifth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application, taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 An application scenario diagram of a job fault takeover method, device, equipment, medium and program product under a unit architecture according to an embodiment of the present application is schematically shown;

[0017] Figure 2 A flowchart of a job fault takeover method under a unit architecture according to an embodiment of the present application is schematically shown;

[0018] Figure 3 Another flowchart of a job fault takeover method under a unit architecture according to an embodiment of the present application is schematically shown;

[0019] Figure 4 A deployment architecture diagram before a job fault takeover under a unit architecture according to an embodiment of the present application is schematically shown;

[0020] Figure 5Fig. 2 schematically illustrates a deployment architecture diagram after job failure takeover under a unitization architecture according to an embodiment of the present application;

[0021] Figure 6 Fig. 3 schematically illustrates a sequence diagram of a job failure takeover method under a unitization architecture according to an embodiment of the present application;

[0022] Figure 7 Fig. 4 schematically illustrates a structure block diagram of a job failure takeover apparatus under a unitization architecture according to an embodiment of the present application; and

[0023] Figure 8 Fig. 5 schematically illustrates a block diagram of an electronic device adapted to implement a job failure takeover method under a unitization architecture according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely illustrative of the present application and in no way limits the scope of the present application. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that one or more embodiments can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0025] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present application. The terms "include" and "have" and the like used herein indicate the presence of the described features, steps, operations, and / or components but do not preclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein, including technical and scientific terms, have the same meanings as those generally understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present description, and should not be interpreted in an idealized or overly formal way.

[0027] In the case of using expressions similar to "at least one of A, B, and C, etc.", it is generally to be interpreted as including one or more of the same unless otherwise specified. For example, "a system having at least one of A, B, and C" should be interpreted as including a system having A alone, a system having B alone, a system having C alone, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc.

[0028] In the technical solutions of the present application, the user information (including but not limited to user personal information, user image information, user device information such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, take necessary security measures, do not violate public order and good customs, and provide corresponding operation portal for user selection authorization or refusal.

[0029] In the scenario of making automated decisions with personal information, the method, device and system provided by the embodiments of the present application all provide corresponding operation portal for the user to select to agree or refuse the automated decision result; if the user chooses to refuse, the expert decision process is entered. The expression "automated decision" here refers to the activity of automatically analyzing, evaluating the behavior habits, interests and hobbies or economic, health, credit status of a person through a computer program and making decisions. The expression "expert decision" here refers to the activity of making decisions by personnel who are engaged in a certain field of work, have special experience, knowledge and skills and reach a certain professional level.

[0030] The embodiments of the present application provide a job failure takeover method, device, equipment, medium and program product under a unit architecture.

[0031] Figure 1 The application scenario diagram of the job failure takeover method, device, equipment, medium and program product under a unit architecture according to the embodiments of the present application is schematically shown.

[0032] As shown in Figure 1 The application scenario 100 according to the embodiments can include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used as a medium to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0033] The user can use the first terminal device 101, the second terminal device 102, the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0035] The server 105 can be a server providing various services, for example, a background management server supporting a website browsed by a user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can perform analysis and the like on received user requests and the like, and feed back a processing result (for example, a webpage, information, or data, or the like, obtained or generated according to a user request) to a terminal device.

[0036] It should be noted that the job failure takeover method under the unit architecture provided in the embodiments of the present application can generally be executed by the server 105. Accordingly, the job failure takeover device under the unit architecture provided in the embodiments of the present application can generally be arranged in the server 105. The job failure takeover method under the unit architecture provided in the embodiments of the present application can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Accordingly, the job failure takeover device under the unit architecture provided in the embodiments of the present application can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0037] It should be understood that the number of terminal devices, networks, and servers in the system shown in FIG. 1 is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks, and servers. Figure 1

[0038] The job failure takeover method under the unit architecture according to the embodiments of the present application will be described in detail below based on the scenario described in Figure 1 Figures 2-6

[0039] Figure 2 An illustrative flowchart of the job failure takeover method under the unit architecture according to the embodiments of the present application is shown.

[0040] As shown in Figure 2 The job failure takeover method under the unit architecture of this embodiment includes operations S210-S230.

[0041] In operation S210, for at least one business job accessing a business system, computing resources corresponding to the business job are divided into a first unit and a second unit.​​​

[0042] In operation S220, in response to the initialization request of the service job, the first health data of the first unit and the second health data of the second unit are acquired.

[0043] In the embodiments of the present application, before the first health data of the first unit and the second health data of the second unit are acquired, the consent or authorization of the user can be obtained. For example, before operation S220, the user can be requested to acquire the first health data of the first unit and the second health data of the second unit. In the case where the user agrees or authorizes the first health data of the first unit and the second health data of the second unit to be acquired, operation S220 is performed.

[0044] In operation S230, in the case where the first health data does not meet the target condition and the second health data meets the target condition, the second unit simultaneously loads the first data shard of the first unit and the second data shard of the second unit to execute the service job in the second unit.

[0045] In the embodiments of the present application, the user can be provided with a corresponding operation portal for the user to select to agree or refuse the automatic decision result. That is, before the second unit simultaneously loads the first data shard of the first unit and the second data shard of the second unit for processing / decision making, the instruction of the user input through the corresponding operation portal to agree or refuse to make the processing / decision making can be obtained. If the user agrees to make the processing / decision making, the second unit simultaneously loads the first data shard of the first unit and the second data shard of the second unit for processing / decision making, that is, operation S230 is performed. If the user refuses to make the processing / decision making, the expert decision making process is entered.

[0046] Figure 3 Another flowchart of the job failure takeover method under the unit architecture according to the embodiments of the present application is schematically shown.

[0047] As shown in Figure 3 In actual application, according to the characteristics and needs of the business system, a plurality of different business modules can be determined, each of which represents a separate batch service job, i.e., business function module 1, business function module 2, …, business function module N. For example, for the core business system of a large bank, it can be divided into savings business module, loan business module and credit card business module, etc. According to each business module, the computing resources can be divided into different units to perform takeover judgment logic processing and select processing data shards in each business module.

[0048] The first unit and the second unit can be understood as logical or physical entities with independent computing and data storage capabilities. For example, in a transaction settlement service module of a financial institution, a part of high-performance server clusters can be designated as the first unit, which is responsible for processing settlement calculation tasks related to stock transactions, such as calculating the profit and loss of each stock transaction, commission fees, and updating the account balance of investors in real time. In addition, a part of server clusters can be arranged to form the second unit as a backup unit, i.e., the first unit and the second unit can be configured in a master-slave mode.

[0049] The embodiments of the present application can provide a common component, i.e., the takeover judgment module, to the various business modules, which is called and judged during the initialization of business operations, and selects and loads corresponding data shard data sources according to the judgment of takeover. Specifically, first, the health data of the current unit (assuming the first unit) and the backup unit (i.e., the second unit) can be obtained through the monitoring component deployed in the system, and then the health data of the current unit is judged. If the target condition is not met, i.e., it is not healthy, the call can be directly ended, and no data shard is loaded, the related operation is empty, and the actual business logic processing is not performed. At the same time, the health data of the backup unit can be judged. If the target condition is met, i.e., it is healthy, the data shards of the two units can be loaded in the second unit at the same time, so that the second unit can take over the business operation originally processed by the first unit, and the continuity of the business is ensured.

[0050] For example, in the corresponding business system of a financial institution, the computing resources corresponding to the account opening business operation can be divided into the first unit and the second unit. The first unit can be responsible for processing the account opening application of individual customers, and the second unit can be used as a backup unit. For example, when a bank expands new customer groups, there will be a large number of individual account opening demands, and the first unit can be responsible for processing these individual account opening information input, identity verification, and account generation tasks from online and offline. When a new account opening business operation is initialized, the system can obtain the health data of the first unit and the second unit at the same time. For example, by monitoring the response time, system resource usage, and other indicators to determine. If the response time of the first unit suddenly becomes longer, the system resource occupation approaches the limit, while the second unit is running normally, responding quickly and having sufficient resources, it can be determined that the health data of the first unit does not meet the preset target condition, while the second unit meets the target condition. At this time, the system can load the account opening data shards (such as customer basic information, account opening application materials, etc.) of the first unit and the data shards of the second unit in the second unit at the same time. Then, the second unit can continue to process the unfinished account opening business, ensuring that new customers can successfully open an account and not affecting the bank's customer expansion business.

[0051] It can be understood that, in view of the batch job data shard isolation under the unit architecture, the timeliness of takeover in the fault scenario is insufficient, an automatic takeover mechanism in a unit fault scenario is provided in the embodiments of the present application, which greatly improves the high availability capability of the batch job under the unit architecture, and realizes automatic takeover and automatic recovery of faults without manual intervention, while retaining the characteristics of business isolation by unit under the unit architecture.

[0052] In the embodiments of the present application, the method further comprises: in the case that the first health data meets the target condition and the second health data does not meet the target condition, loading the first data shard and the second data shard in the first unit at the same time to execute the business job in the first unit; or in the case that the first health data and the second health data both do not meet the target condition, terminating the execution of the business job.

[0053] When the health data of the first unit meets the preset target condition and the health data of the second unit does not meet the target condition, it indicates that the first unit can work normally and the second unit has a fault. At this time, the system can load the first data shard and the second data shard in the first unit at the same time to execute the business job in the first unit.

[0054] Taking a loan approval system in the financial industry as an example, the first unit is responsible for processing personal small loan approval business, and the second unit originally serves as a backup unit and can take over the business when the first unit fails. If in the running process, the performance indicators of the first unit all meet the target condition, and the health data of the second unit does not meet the target condition due to hardware failure or software exception. At this time, the loan approval related data (second data shard) originally planned to be processed in the second unit can also be loaded into the first unit. The first unit can also be responsible for the business originally processed by the second unit (processing the second data shard) while completing the personal small loan approval business (processing the first data shard) it is responsible for, ensuring that the loan approval business will not be interrupted due to the failure of the second unit, and ensuring the continuity of financial services.

[0055] When the health data of the first unit and the second unit both do not meet the target condition, it indicates that both units have failed and cannot normally execute the business job. In this case, the system can terminate the execution of the business job. At the same time, the system can trigger an early warning mechanism to notify the technical personnel to promptly troubleshoot and repair, so as to restore the normal operation of the system as soon as possible.

[0056] When the health data of the first unit and the second unit both meet the target condition, it indicates that both units can work normally. At this time, the first shard data can be loaded in the first unit and the second shard data can be loaded in the second unit to improve processing efficiency and professionalism through cooperative work.

[0057] It can be understood that, by considering various cases under different unit health data, a comprehensive fault handling method is provided. Whether it is a single unit failure or a simultaneous failure of two units, a reasonable handling decision can be made to minimize the business interruption time and ensure the continuity of the business.

[0058] In the embodiments of the present application, the method further comprises: storing the first health data of the first unit and the second health data of the second unit into a storage architecture deployed in multiple data centers, wherein each data center independently stores full health data, and data synchronization is realized through a semi-synchronous replication mechanism without automatic degradation mode.

[0059] A distributed parameter storage module can be built by constructing components of a distributed system. This module can be a common module shared at the unit level to store the health data of each unit. In terms of architecture deployment, a park-level three-active deployment can be performed, that is, three active instances are simultaneously run in different parks (i.e., data centers), which has a park-level high-availability capability. Each data center can independently store full health data, thereby effectively avoiding the loss or unavailability of data due to the failure of a single data center.

[0060] For example, in a business system in the financial industry, multiple data centers can be established in different regions across the country. Each data center stores the health data of the first unit and the second unit completely, so that even if a natural disaster or network failure occurs in a certain region, other data centers can still provide data services normally, ensuring that the monitoring of unit health data by the system is not affected.

[0061] It should be noted that the first health data of the first unit and the second health data of the second unit are only examples, and can also include the health data of other multiple units.

[0062] The storage layer can use a cluster deployment of MySQL (semi-synchronous non-degradation), to ensure data consistency in each park. MySQL cluster can ensure data synchronization between multiple nodes through a semi-synchronous replication mechanism, and the non-automatic degradation mode can further enhance data consistency. In the face of network fluctuations or partial node failures, the system can still maintain data consistency, avoiding the problem of data conflict or loss.

[0063] In the job failure takeover process, each business module can obtain the health data of the first unit and the second unit through the parameter storage module, and then make job allocation and failure takeover decisions according to the preset rules. For example, when a business module needs to determine whether to transfer a job, it can obtain unit health data from the parameter storage module. If it is found that the first unit is unhealthy and the second unit is healthy, the corresponding failure takeover process can be triggered.

[0064] It can be understood that the combination of the multi-data center storage architecture and the semi-synchronous replication mechanism ensures the reliable storage and consistent synchronization of the unit health data. Even in the case of complex network environment and partial data center failure, the accuracy and integrity of the data can be guaranteed, thereby providing a solid data foundation for job failure takeover.

[0065] On the basis of the above-mentioned embodiments, in this embodiment, the first health data of the first unit is obtained, including: obtaining the number of live containers of the first unit; when the number of live containers of the first unit is less than or equal to a first preset threshold, updating the abnormal count to m+1, wherein m represents the current abnormal count value and m≥0; and when the abnormal count is greater than or equal to a second preset threshold, determining that the first health data of the first unit does not meet the target condition.

[0066] The unit alive detection module can be used asynchronously in the background of each unit node to monitor the number of live containers of the unit in real time, and to determine the health status of the unit according to the monitoring result. An alive detection interface can be configured at the SLB (load balancer) level of each unit, through which the number of live containers of the unit is returned, providing data support for subsequent health status determination. The container is the basic unit of the business application, and each unit can be composed of multiple containers. The number of live containers can reflect the current running capacity and resource availability of the unit.

[0067] Continuing to refer to Figure 3 , first, the health alive detection interface of the first unit SLB can be called to obtain the number of live containers of the first unit at present, so as to understand the resource usage and running status of the first unit in real time. If the number of live containers of the first unit is equal to the first preset threshold (such as 0), it indicates that the first unit may have failed and cannot normally provide services. At this time, the number of health check failures (i.e. abnormal count) can be increased by 1. The abnormal count is an index for recording the continuous failure of the first unit, and can reflect the stability of the first unit in a period of time. If the number of health check failures is greater than or equal to the second preset threshold (such as 3), i.e. does not meet the target condition, it indicates that the first unit has been in an unhealthy state for a long time and cannot undertake the business takeover task. At this time, the first unit health data can be set as unhealthy by calling the parameter storage module interface. If the number of health check failures is less than the second preset threshold, it indicates that it cannot be determined whether the health data of the first unit meets the target condition, and the next alive detection can be used to further determine it.

[0068] It should be understood that the second unit health monitoring in the embodiments of the present application corresponds to the first unit monitoring part, and the specific implementation details are the same, which will not be repeated here.

[0069] It can be understood that by acquiring the number of live containers and combining the abnormal count for judgment, the health status of the unit can be more accurately monitored. Compared with single index monitoring, this way comprehensively considers the short-term abnormality and long-term stability of the unit, and avoids misjudgment of the unit state due to temporary fluctuations.

[0070] In the embodiments of the present application, the method further comprises: when the number of live containers of the first unit is greater than a first preset threshold, resetting the abnormal count to an initial value, and determining that the first health data of the first unit satisfies a target condition; and loading the first data shard for processing by the first unit, and loading the second data shard for processing by the second unit.

[0071] In the unit alive detection module, it can be judged whether the number of live containers of the first unit is less than or equal to a first preset threshold. When the number of live containers of the first unit is greater than the first preset threshold, it means that the first unit is currently in a normal running state. At this time, the abnormal count can be reset to an initial value (the initial value can be 0), that is, the number of health check failures accumulated due to temporary abnormalities is cleared, and the first health data of the first unit can be re-identified as satisfying the target condition. Then, the parameter storage module interface can be called to set the first unit health data to healthy.

[0072] After determining the health of the first unit, the first unit can reload the first data shard for processing, and the second unit can continue to load the second data shard for processing.

[0073] Exemplarily, in a certain commercial bank intelligent investment consultant business system, a unit architecture is adopted, the computing resources are divided into a first unit responsible for stock investment consultant business and a second unit responsible for fund investment consultant business, and are deployed in a containerized manner. Initially, the first unit has a sudden network failure, the number of live containers is equal to the first preset threshold 0, and the abnormal count also changes from 0 to 1. After 20 seconds, the second alive detection is performed, and the returned number of live containers is still 0, and the abnormal count increases to 2. In the third alive detection, the abnormal count reaches 3, triggering the health state change, and the first unit is set to "unhealthy". At this time, the job scheduling can immediately freeze its new request, and perform dynamic expansion, that is, migrate the stock business data shard to the second unit, and the second unit loads the original fund business data shard and the newly taken over stock business data shard. In this process, the business switching can be completed within 30 seconds to ensure the availability of the service. After 15 minutes, the first unit is repaired, the number of live containers returns to 5, which is greater than the first preset threshold, and the abnormal count can be reset to 0, and the health state returns to "healthy". At this time, the job scheduling can start load balancing to migrate part of the traffic back to the first unit, that is, the first unit can continue to process the stock business data shard, and the second unit automatically shrinks.

[0074] It can be understood that when the number of live containers of the unit is restored to be above the healthy threshold, by resetting the abnormal count to the initial value in time, the system misjudgment of the unit being unhealthy for a long time due to temporary failure fluctuation can be effectively avoided, unnecessary service switching and resource waste can be avoided, and the system can be ensured to quickly recover to the normal monitoring state when facing temporary failure, and the overall stability can be maintained. In addition, the parallel processing mode of the first unit and the second unit can fully utilize the computing resources of the system and improve the business processing efficiency.

[0075] In the embodiment of the present application, the method further comprises: after completing the detection of the current first health data, sleeping for a preset sleep period; and in response to the end of the sleep period, triggering the first health data to be probed alive.

[0076] After completing the detection of the current first health data, the system can sleep for a preset sleep period. The length of the sleep period can be set according to the performance requirements and the monitoring frequency of the business system. For example, in the embodiment, the sleep period can be set to 20 seconds, that is, after completing the health data detection once, the system can pause for 20 seconds before the next detection.

[0077] When the sleep period ends, the system can automatically trigger the next probing alive process. Taking 20 seconds as a probing alive period as an example, the system can detect the health indicators such as the number of live containers of the first unit every 20 seconds, make a health determination according to the preset rules, and update the abnormal count and the health state data according to the determination result, so as to realize the continuous monitoring of the unit health state, so as to timely grasp the running situation of the unit.

[0078] It can be understood that the periodic probing alive mechanism ensures the real-time and continuity of state monitoring, and ensures that the unit can be timely discovered and corresponding measures can be taken when the unit fails.

[0079] On the basis of the above-mentioned embodiment, in the embodiment, the loading of the first data shard of the first unit and the second data shard of the second unit in the second unit to execute the business job in the second unit comprises: promoting the backup library corresponding to the first unit to a new master library of the second unit, taking over the first data shard of the original master library of the first unit; and configuring the batch job node of the second unit, connecting the original master library of the second unit to process the second data shard of the second unit, and connecting the new master library after promotion to process the first data shard of the first unit.

[0080] Figure 4 and Figure 5 The deployment architecture diagrams before and after the job failure takeover under the unit architecture according to the embodiments of the present application are respectively and illustratively shown.

[0081] As Figure 4 shown, the first unit (i.e. unit one) and the second unit (i.e. unit two) adopt a master-standby architecture for data storage. Among them, SET1 is the master database of unit one, storing the core business data of unit one, and SET1' is its backup database (cold standby) in unit two, which does not participate in business processing under normal circumstances and only serves as a data redundancy backup. Similarly, SET2 is the master database of unit two, storing the business data of unit two, and SET2' is its backup database in unit one. Under the condition that both unit one and unit two are healthy, the batch job nodes of each unit only process the data shards of the unit. For example, the batch nodes of unit one focus on processing data in SET1 to complete the business logic of unit one; the batch nodes of unit two process data in SET2 to ensure the normal operation of unit two business.

[0082] As Figure 5 shown, when unit one fails, the backup database SET1' corresponding to the first unit can be promoted to the master database to assume the data service responsibilities originally handled by SET1. At this time, the batch nodes of unit two can be adjusted accordingly, and SET1' and SET2 are connected to process all data shards of unit one and unit two.

[0083] Exemplarily, in a financial risk control system, unit one is originally responsible for real-time monitoring of user transaction risks, and when unit one fails, SET1' becomes the new master database, storing the transaction risk data of unit one. At this time, the batch nodes of unit two not only need to process the customer credit evaluation data shards in SET2, but also need to connect SET1' to process the transaction risk monitoring data shards of unit one, to ensure that the entire financial risk control business can continue to run stably.

[0084] It can be understood that through the role conversion of master-standby databases and the flexible configuration of batch job nodes, the business takeover can be quickly realized when the unit fails, ensuring the high availability of the system.

[0085] Figure 6 The sequence diagram of the job failure takeover method under the unit architecture according to the embodiments of the present application is schematically shown.

[0086] As Figure 6As shown, when a container of unit one is offline, the health probe thread of unit two node can call the SLB probe interface to obtain the number of live containers of unit one. If the SLB interface returns the number of live containers as 0 for three times in succession, the unit two node can call the parameter storage module interface to set the health status of unit one as unhealthy. Then, when the unit two node task is executed, the parameter storage module interface is called to return the unhealthy state of unit one, at which time the unit two node loads the data shards of unit one and unit two for processing. After unit one recovers and the containers are online again, the health probe thread of unit two node calls the SLB probe interface to obtain the number of live containers of unit one again. When the returned number of live containers is not 0, the unit two node calls the parameter storage module interface to set the health status of unit one as healthy. After that, when the unit two node task is executed, the parameter storage module interface is called to return that unit one is healthy, and only the data shards of unit two are loaded for processing. At the same time, when the unit one node task is executed, the parameter storage module interface is called to return that unit one is healthy, and the data shards of unit one are normally loaded for processing.

[0087] Based on the job fault takeover method under the unit architecture described above, the application further provides a job fault takeover device under a unit architecture. The following will be described in detail in combination with Figure 7 the device.

[0088] Figure 7 The structure block diagram of the job fault takeover device under the unit architecture according to the embodiment of the application is schematically shown.

[0089] As Figure 7 shown, the job fault takeover device 700 under the unit architecture of the embodiment includes a division module 710, an acquisition module 720, and a job module 730.

[0090] The division module 710 is configured to divide computing resources corresponding to at least one business job of an access business system into a first unit and a second unit. In an embodiment, the division module 710 can be configured to perform the operation S210 described above, and details are not described herein again.

[0091] The acquisition module 720 is configured to acquire first health data of the first unit and second health data of the second unit in response to an initialization request of the business job. In an embodiment, the acquisition module 720 can be configured to perform the operation S220 described above, and details are not described herein again.

[0092] The job module 730 is configured to, in a case where the first health data does not satisfy the target condition and the second health data satisfies the target condition, load the first data shard of the first unit and the second data shard of the second unit simultaneously at the second unit to execute the business job at the second unit. In an embodiment, the job module 730 can be configured to perform the operation S230 described above, and details are not described herein again.

[0093] According to an embodiment of the present application, the job module 730 can also be configured to, in a case where the first health data satisfies the target condition and the second health data does not satisfy the target condition, load the first data shard and the second data shard simultaneously at the first unit to execute the business job at the first unit, or in a case where neither the first health data nor the second health data satisfies the target condition, terminate the execution of the business job.

[0094] According to an embodiment of the present application, the obtaining module 720 is specifically configured to: obtain the number of live containers of the first unit; update the abnormality count to m+1 when the number of live containers of the first unit is less than or equal to a first preset threshold, where m represents a current abnormality count value and m≥0; and determine that the first health data of the first unit does not satisfy the target condition when the abnormality count is greater than or equal to a second preset threshold.

[0095] According to an embodiment of the present application, the obtaining module 720 can also be configured to: reset the abnormality count to an initial value when the number of live containers of the first unit is greater than the first preset threshold, and determine that the first health data of the first unit satisfies the target condition; and load the first data shard for processing at the first unit and load the second data shard for processing at the second unit.

[0096] According to an embodiment of the present application, the obtaining module 720 can also be configured to: sleep for a preset sleep period after completing the detection of the current first health data; and in response to the end of the sleep period, trigger the live detection of the first health data.

[0097] According to an embodiment of the present application, the obtaining module 720 can also be configured to: store the first health data of the first unit and the second health data of the second unit into a storage architecture deployed by a plurality of data centers, where each data center independently stores full health data, and data synchronization is achieved through a semi-synchronous replication mechanism in a non-automatic degradation mode.

[0098] According to an embodiment of the present application, the job module 730 is specifically configured to promote the backup database corresponding to the first unit as a new master database of the second unit, take over the first data shard of the original master database of the first unit, and configure the batch job node of the second unit, connect the original master database of the second unit to process the second data shard of the second unit, and connect the new master database after promotion to process the first data shard of the first unit.

[0099] According to an embodiment of the present application, any of the dividing module 710, the obtaining module 720 and the job module 730 can be combined in one module, or any of the modules can be split into multiple modules. Alternatively, at least part of the function of one or more of the modules can be combined with at least part of the function of the other modules, and implemented in one module. According to an embodiment of the present application, at least one of the dividing module 710, the obtaining module 720 and the job module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of hardware or firmware that can be integrated or packaged, or implemented in any one of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the dividing module 710, the obtaining module 720 and the job module 730 can be at least partially implemented as a computer program module that can perform the corresponding function when the computer program module is run.

[0100] Figure 8 A block diagram of an electronic device suitable for implementing the job failure takeover method under the unit architecture according to an embodiment of the present application is schematically shown.

[0101] As shown in Figure 8 The electronic device 800 according to an embodiment of the present application includes a processor 801 that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 802 or loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), and the like. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present application.

[0102] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via the bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present application by executing the programs in the ROM 802 and / or the RAM 803. It should be noted that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method flow according to the embodiments of the present application by executing the programs stored in the one or more memories.

[0103] According to the embodiments of the present application, the electronic device 800 can further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 can further include one or more of the following components connected to the input / output (I / O) interface 805: an input part 806 including a keyboard, a mouse, and the like; an output part 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part 808 including a hard disk, and the like; and a communication part 809 including a network interface card such as a LAN card, a modem, and the like. The communication part 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 810 as necessary, so that a computer program read therefrom is installed in the storage part 808 as necessary.

[0104] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present application is implemented.

[0105] According to an embodiment of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, can include but is not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this application, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, in an embodiment of the present application, a computer readable storage medium can include the ROM 802 and / or the RAM 803 described above, and / or one or more other memories that are not the ROM 802 and the RAM 803.

[0106] Embodiments of the present application also include a computer program product, which includes a computer program containing program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the job failure takeover method under the unit architecture provided by the embodiments of the present application.

[0107] The above functions defined in the system / device of the embodiments of the present application are performed when the computer program is executed by the processor 801. According to an embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0108] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of signals on a network medium, and be downloaded and installed through the communication part 809, and / or installed from the detachable medium 811. The program codes contained in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

[0109] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the detachable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiments of the present application are performed. According to an embodiment of the present application, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0110] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. Programming languages include, but are not limited to, Java, C++, python, "C", or the like. Program code can execute entirely on a user's computing device, partly on the user's device, as a stand-alone software package, partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0111] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0112] Those skilled in the art will appreciate that features recited in the various embodiments of the present application can be combined and / or integrated in various combinations, even if such combinations have not been explicitly recited in the present application. In particular, the features recited in the various embodiments of the present application can be combined and / or integrated in various combinations, without departing from the spirit and teachings of the present application. All such combinations are within the scope of the present application.

Claims

1. A method for job fault takeover under a unitized architecture, characterized in that, The method includes: For at least one business job in the access business system, the computing resources corresponding to the business job are divided into a first unit and a second unit. In response to the initialization request of the business operation, the first health data of the first unit and the second health data of the second unit are obtained; and If the first health data does not meet the target conditions, but the second health data does meet the target conditions, the first data fragment of the first unit and the second data fragment of the second unit are loaded simultaneously in the second unit to execute the business operation in the second unit.

2. The method according to claim 1, characterized in that, The method further includes: If the first health data meets the target condition, but the second health data does not meet the target condition, the first data shard and the second data shard are loaded simultaneously in the first unit to execute the business operation in the first unit; or If neither the first health data nor the second health data meets the target conditions, the execution of the business operation shall be terminated.

3. The method according to claim 1, characterized in that, Obtaining the first health data of the first unit includes: Obtain the number of surviving containers in the first unit; When the number of surviving containers in the first unit is less than or equal to a first preset threshold, the anomaly count is updated to m+1, where m represents the current anomaly count value and m≥0; and When the abnormal count is greater than or equal to the second preset threshold, it is determined that the first health data of the first unit does not meet the target condition.

4. The method according to claim 3, characterized in that, The method further includes: When the number of surviving containers in the first unit exceeds a first preset threshold, the anomaly count is reset to its initial value, and the first health data of the first unit is determined to meet the target condition; and The first data fragment is loaded and processed in the first unit, and the second data fragment is loaded and processed in the second unit.

5. The method according to claim 3 or 4, characterized in that, The method further includes: After completing the detection of the first health data, hibernate within a preset hibernation period; and In response to the end of the dormancy period, the detection of the first health data is triggered.

6. The method according to claim 1 or 2, characterized in that, The method further includes: The first health data of the first unit and the second health data of the second unit are stored in a storage architecture deployed in multiple data centers. Each data center independently stores the full amount of health data, and data synchronization is achieved through a semi-synchronous replication mechanism without automatic degradation mode.

7. The method according to claim 1 or 2, characterized in that, The step of simultaneously loading the first data fragment of the first unit and the second data fragment of the second unit in the second unit to execute the business operation in the second unit includes: Promote the backup database corresponding to the first unit to the new primary database of the second unit, and take over the first data shard of the original primary database of the first unit; and Configure the batch job node of the second unit to connect to the original master database of the second unit to process the second data shard of the second unit, and connect to the promoted new master database to process the first data shard of the first unit.

8. A work failure takeover device under a modular architecture, characterized in that, The device includes: The partitioning module is used to partition the computing resources corresponding to at least one business job in the access business system into a first unit and a second unit. The acquisition module is configured to, in response to the initialization request of the business operation, acquire the first health data of the first unit and the second health data of the second unit; and The job module is used to load the first data fragment of the first unit and the second data fragment of the second unit simultaneously in the second unit when the first health data does not meet the target conditions and the second health data meets the target conditions, so as to execute the business job in the second unit.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images