Batch program processing method, device and system based on microservice architecture

Abstract

The present application discloses a batch program processing method, device and system based on a microservice architecture, which can be applied to distributed fields or financial fields. The batch startup server runs the batch startup script according to the batch running time configured by the batch scheduling server, initiates the microservice calling program through the running batch startup script, and sends the batch program identifier to the microservice calling program. Through the microservice calling program, the batch program related information corresponding to the batch program identifier is obtained from the database server, and the corresponding target batch program is called through the microservice registration center server according to the batch program related information. The present invention adopts a microserver architecture to realize that a batch startup script communicates with each batch program through the microservice registration center server, thereby realizing the distributed deployment of batch programs, and then realizing distributed multi-active, and improving the reliability of batch program processing.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a batch program processing method, device and system based on a microservice architecture. Background Art

[0002] In the prior art, batch processing is usually performed by Figure 1 In the system environment shown, there are multiple batch programs in the system environment, and each batch program is connected to a batch scheduling server (such as crontab) through a batch startup script. For example, the system environment includes: batch program 1, batch program 2 and batch program 3. Batch program 1 is connected to batch startup script 1, batch program 2 is connected to batch startup script 2, and batch program 3 is connected to batch startup script 3. When processing the target batch program, the batch scheduling server periodically calls up the corresponding batch program through the batch startup script connected to the target batch program.

[0003] Although, Figure 1 The system environment shown in the figure is relatively simple, and the batch programs are independent of each other. However, when there are many batch programs, since the batch startup scripts and batch programs can only be deployed on a single server, distributed deployment cannot be performed, and distributed multi-active cannot be achieved. Summary of the invention

[0004] In view of this, the present invention discloses a batch program processing method, device and system based on a microservice architecture to achieve distributed deployment of batch programs.

[0005] A batch program processing method based on a microservice architecture is applied to batch start servers, and the batch program processing method includes:

[0006] Run the batch startup script according to the batch running time configured by the batch scheduling server;

[0007] Initiate a microservice invocation program by running the batch startup script, and send a batch program identifier to the microservice invocation program, wherein the batch program identifier is pre-configured by the batch scheduling server;

[0008] Invoke a program through the microservice and obtain batch program related information corresponding to the batch program identifier from a database server;

[0009] The corresponding target batch program is called through the microservice registration center server according to the batch program related information.

[0010] Optionally, the batch program related information includes: file directory, file format, user name and password.

[0011] Optionally, the microservice registration center server includes: ZooKeeper.

[0012] Optionally, also include:

[0013] When a new batch program is added, a new batch program identifier is added to the batch startup script;

[0014] The newly added batch program related information corresponding to the newly added batch program identifier is added in the database server through the microservice invoking program.

[0015] Optionally, the batch program is identified as a month report.

[0016] A batch program processing device based on a microservice architecture is applied to batch start servers, and the batch program processing device includes:

[0017] The running unit is used to run the batch startup script according to the batch running time configured by the batch scheduling server;

[0018] A program initiating unit, configured to initiate a microservice initiating program by running the batch initiating script, and send a batch program identifier to the microservice initiating program, wherein the batch program identifier is pre-configured by the batch scheduling server;

[0019] An information acquisition unit, configured to call up a program through the microservice and acquire batch program related information corresponding to the batch program identifier from a database server;

[0020] The program calling unit is used to call the corresponding target batch program through the microservice registration center server according to the batch program related information.

[0021] Optionally, also include:

[0022] An identification adding unit, used for adding a new batch program identification to the batch startup script when a new batch program is added;

[0023] The information adding unit is used to add the newly added batch program related information corresponding to the newly added batch program identifier in the database server through the microservice calling program.

[0024] A batch program processing system based on a microservice architecture, comprising: a batch scheduling server, a batch startup server, a database server, a microservice registration center server and at least one microservice server, each of which stores a batch program;

[0025] The batch scheduling server is connected to the batch startup server, and the batch scheduling server is used to configure batch running time, batch startup scripts and batch program identifiers in the batch startup server;

[0026] The batch startup server is connected to the database server and the microservice registration center server respectively, and the batch startup server includes the batch program processing device described above;

[0027] The database server is used to store the corresponding relationship between the batch program identifier and the batch program related information;

[0028] The microservice registration center server is connected to each of the microservice servers and is used to call the corresponding target batch program according to the batch program related information sent by the batch startup server.

[0029] An electronic device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the batch program processing method described above.

[0030] A computer-readable storage medium stores a computer program for executing the batch program processing method described above.

[0031] It can be seen from the above technical scheme that the present invention discloses a batch program processing method, device and system based on microservice architecture, wherein the batch startup server runs the batch startup script according to the batch running time configured by the batch scheduling server, initiates the microservice calling program through the running batch startup script, and sends the batch program identifier to the microservice calling program, obtains the batch program related information corresponding to the batch program identifier from the database server through the microservice calling program, and calls the corresponding target batch program through the microservice registration center server according to the batch program related information. The present invention adopts the microserver architecture to realize the communication between a batch startup script and each batch program through the microservice registration center server, thereby realizing the distributed deployment of the batch program, and then realizing the distributed multi-active, and improving the reliability of batch program processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without paying creative work.

[0033] Figure 1A schematic diagram of a system environment for batch program processing in the prior art;

[0034] Figure 2 A flow chart of a batch program processing method based on a microservice architecture disclosed in an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the structure of a batch program processing device based on a microservice architecture disclosed in an embodiment of the present invention;

[0036] Figure 4 A schematic diagram of the structure of a batch program processing system based on a microservice architecture disclosed in an embodiment of the present invention;

[0037] Figure 5 The present invention is a schematic diagram of the hardware structure of an electronic device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] It should be noted that the batch program processing method, device and system based on microservice architecture provided by the present invention can be used in distributed fields or financial fields. The above is only an example and does not limit the application field of the batch program processing method, device and system based on microservice architecture provided by the present invention.

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The embodiment of the present invention discloses a batch program processing method, device and system based on a microservice architecture. The batch startup server runs a batch startup script according to the batch running time configured by the batch scheduling server, initiates a microservice calling program through the running batch startup script, and sends the batch program identifier to the microservice calling program. The microservice calling program obtains batch program related information corresponding to the batch program identifier from the database server, and calls the corresponding target batch program through the microservice registration center server according to the batch program related information. The present invention adopts a microserver architecture to realize a batch startup script to communicate with each batch program through the microservice registration center server, thereby realizing the distributed deployment of the batch program, and then realizing distributed multi-active, and improving the reliability of batch program processing.

[0041] See also Figure 2 , a flow chart of a batch program processing method based on a microservice architecture disclosed in an embodiment of the present invention, the method is applied to batch start servers, and the batch program processing method includes:

[0042] Step S101, running a batch startup script according to the batch running time configured by the batch scheduling server;

[0043] In actual applications, the batch start server is connected to the batch scheduling server, and the batch scheduling server can configure the batch running time in the batch start server through the batch scheduling tool installed inside it. In addition to configuring the batch running time, the batch scheduling server also configures the batch start script and the batch program identifier in the batch start server, and the batch program identifier can be a month report.

[0044] It should be particularly noted that there is only one batch startup script in the present invention, and the batch startup script is used for different batch programs, but different batch programs correspond to different batch program identifiers in the batch startup script.

[0045] Step S102: Initiate a microservice invocation program by running the batch invocation script, and send a batch program identifier to the microservice invocation program;

[0046] Wherein, the batch program identifier is pre-configured by the batch scheduling server.

[0047] In this embodiment, in addition to the batch startup script, the batch startup server also has a microservice invocation program. After the microservice invocation program is initiated by the batch startup script, it will receive the batch program identifier sent by the batch startup script.

[0048] Step S103, calling up a program through the microservice, and obtaining batch program related information corresponding to the batch program identifier from a database server;

[0049] The database server has a database or file storing the correspondence between batch program identifiers and batch program related information.

[0050] Batch program related information includes but is not limited to: file directory, file format, user name, password and other parameters.

[0051] Step S104: calling the corresponding target batch program through the microservice registration center server according to the batch program related information.

[0052] Preferably, the microservice registry server includes ZooKeeper.

[0053] ZooKeeper is a distributed, open source distributed application coordination service. It is an open source implementation of Google's Chubby and an important component of Hadoop and Hbase. It is a software that provides consistency services for distributed applications. The functions it provides include: configuration maintenance, domain name services, distributed synchronization, group services, etc. The goal of ZooKeeper is to encapsulate complex and error-prone key services and provide users with simple and easy-to-use interfaces and efficient and stable systems.

[0054] In summary, the present invention discloses a batch program processing method based on a microservice architecture, wherein a batch startup server runs a batch startup script according to a batch running time configured by a batch scheduling server, initiates a microservice call-up program through the running batch startup script, and sends a batch program identifier to the microservice call-up program, obtains batch program related information corresponding to the batch program identifier from a database server through the microservice call-up program, and calls a corresponding target batch program through a microservice registration center server according to the batch program related information. The present invention adopts a microserver architecture to realize a batch startup script communicating with each batch program through a microservice registration center server, thereby realizing distributed deployment of batch programs, and further realizing distributed multi-active, and improving the reliability of batch program processing.

[0055] In addition, in addition to the code information, the batch startup script in the present invention also includes a batch program identifier, based on which the corresponding target batch program can be determined. Therefore, the present invention facilitates the development, operation, and management of batch programs.

[0056] To further optimize the above embodiment, the batch program processing method may further include:

[0057] When a new batch program is added, a new batch program identifier is added to the batch startup script;

[0058] The newly added batch program related information corresponding to the newly added batch program identifier is added in the database server through the microservice invoking program.

[0059] In actual applications, when a new batch program is added, a new batch program identifier is added to the batch startup script, and then the new batch program related information corresponding to the new batch program identifier is added to the database server through the microservice calling program, thereby completing the configuration of the new batch program in the batch startup service.

[0060] Corresponding to the above method embodiments, the present invention also discloses a batch program processing device based on a microservice architecture.

[0061] See also Figure 3, a structural diagram of a batch program processing device based on a microservice architecture disclosed in an embodiment of the present invention, the device is applied to batch start servers, and the batch program processing device includes:

[0062] The running unit 201 is used to run the batch startup script according to the batch running time configured by the batch scheduling server;

[0063] In actual applications, the batch start server is connected to the batch scheduling server, and the batch scheduling server can configure the batch running time in the batch start server through the batch scheduling tool installed inside it. In addition to configuring the batch running time, the batch scheduling server also configures the batch start script and the batch program identifier in the batch start server, and the batch program identifier can be a month report.

[0064] It should be particularly noted that there is only one batch startup script in the present invention, and the batch startup script is used for different batch programs, but different batch programs correspond to different batch program identifiers in the batch startup script.

[0065] A program initiating unit 202 is used to initiate a microservice initiating program by running the batch startup script, and send a batch program identifier to the microservice initiating program, wherein the batch program identifier is pre-configured by the batch scheduling server;

[0066] In this embodiment, in addition to the batch startup script, the batch startup server also has a microservice invocation program. After the microservice invocation program is initiated by the batch startup script, it will receive the batch program identifier sent by the batch startup script.

[0067] An information acquisition unit 203 is used to call up a program through the microservice and acquire batch program related information corresponding to the batch program identifier from a database server;

[0068] The database server has a database or file storing the correspondence between batch program identifiers and batch program related information.

[0069] Batch program related information includes but is not limited to: file directory, file format, user name, password and other parameters.

[0070] The program calling unit 204 is used to call the corresponding target batch program through the microservice registration center server according to the batch program related information.

[0071] Preferably, the microservice registry server includes ZooKeeper.

[0072] In summary, the present invention discloses a batch program processing device based on a microservice architecture, wherein a batch startup server runs a batch startup script according to a batch running time configured by a batch scheduling server, initiates a microservice call-up program through the running batch startup script, and sends a batch program identifier to the microservice call-up program, obtains batch program related information corresponding to the batch program identifier from a database server through the microservice call-up program, and calls the corresponding target batch program through a microservice registration center server according to the batch program related information. The present invention adopts a microserver architecture to realize a batch startup script communicating with each batch program through a microservice registration center server, thereby realizing distributed deployment of batch programs, and further realizing distributed multi-active, and improving the reliability of batch program processing.

[0073] In addition, in addition to the code information, the batch startup script in the present invention also includes a batch program identifier, based on which the corresponding target batch program can be determined. Therefore, the present invention facilitates the development, operation, and management of batch programs.

[0074] To further optimize the above embodiment, the batch program processing device may further include:

[0075] An identification adding unit, used for adding a new batch program identification to the batch startup script when a new batch program is added;

[0076] The information adding unit is used to add the newly added batch program related information corresponding to the newly added batch program identifier in the database server through the microservice calling program.

[0077] In actual applications, when a new batch program is added, a new batch program identifier is added to the batch startup script, and then the new batch program related information corresponding to the new batch program identifier is added to the database server through the microservice calling program, thereby completing the configuration of the new batch program in the batch startup service.

[0078] Corresponding to the above embodiments, the present invention also discloses a batch program processing system based on a microservice architecture.

[0079] See also Figure 4 , a structural diagram of a batch program processing system based on a microservice architecture disclosed in an embodiment of the present invention, the batch program processing system includes: a batch scheduling server 301, a batch startup server 302, a database server 303, a microservice registration center server 304 and at least one microservice server 305, and each of the microservice servers 305 stores batch programs.

[0080] in:

[0081] The batch scheduling server 301 is connected to the batch startup server 302 , and the batch scheduling server 301 is used to configure batch running time, batch startup scripts and batch program identifiers in the batch startup server 302 .

[0082] The batch startup server 302 is connected to the database server 303 and the microservice registration center server 304 respectively, and the batch startup server 302 includes the above-mentioned batch program processing device.

[0083] The database server 303 is used to store the correspondence between the batch program identifier and the batch program related information.

[0084] The microservice registration center server 304 is connected to each of the microservice servers 305 and is used to call the corresponding target batch program according to the batch program related information sent by the batch startup server 301.

[0085] It should be noted that the process of batch starting the server 302 to perform batch program processing can refer to the working principle of the batch program processing device, which will not be repeated here.

[0086] In summary, the present invention discloses a batch program processing system based on a microservice architecture, the processing system includes: a batch scheduling server 301, a batch startup server 302, a database server 303, a microservice registration center server 304 and at least one microservice server 305, each of the microservice servers 305 stores a batch program, the batch startup server 302 runs a batch startup script according to the batch running time configured by the batch scheduling server 301, initiates a microservice call-up program through the running batch startup script, and sends a batch program identifier to the microservice call-up program, obtains batch program related information corresponding to the batch program identifier from the database server 303 through the microservice call-up program, and calls the corresponding target batch program through the microservice registration center server 304 according to the batch program related information. The present invention adopts a microserver architecture to realize a batch startup script to communicate with each batch program through the microservice registration center server, thereby realizing the distributed deployment of the batch program, and then realizing distributed multi-active, and improving the reliability of batch program processing.

[0087] In addition, in addition to the code information, the batch startup script in the present invention also includes a batch program identifier, based on which the corresponding target batch program can be determined. Therefore, the present invention facilitates the development, operation, and management of batch programs.

[0088] Corresponding to the above embodiment, Figure 5 As shown, the present invention also provides an electronic device, which may include: a processor 1 and a memory 2;

[0089] The processor 1 and the memory 2 communicate with each other via a communication bus 3;

[0090] Processor 1, for executing computer programs;

[0091] Memory 2, for storing computer programs;

[0092] The processor 1 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0093] The memory 2 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0094] The processor executes the computer program to implement the following functions:

[0095] Run the batch startup script according to the batch running time configured by the batch scheduling server;

[0096] Initiate a microservice invocation program by running the batch startup script, and send a batch program identifier to the microservice invocation program, wherein the batch program identifier is pre-configured by the batch scheduling server;

[0097] Invoke a program through the microservice and obtain batch program related information corresponding to the batch program identifier from a database server;

[0098] The corresponding target batch program is called through the microservice registration center server according to the batch program related information.

[0099] Corresponding to the above-mentioned embodiment, the present invention further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the batch program processing method.

[0100] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0101] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0102] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A batch program processing method based on microservice architecture, characterized in that: Applied to batch startup of servers, the batch program processing method includes: Run the batch startup script according to the batch running time configured by the batch scheduling server; The microservice invocation program is initiated by running the batch invocation script, and the batch program identifier is sent to the microservice invocation program, wherein the batch program identifier is pre-configured by the batch scheduling server; the batch invocation script and the batch program identifier are configured in the batch invocation server, and the batch invocation script is used for different batch programs, and different batch programs correspond to different batch program identifiers in the batch invocation script; the batch invocation script includes code information and batch program identifier, and the corresponding target batch program is determined based on the batch program identifier; Invoke a program through the microservice and obtain batch program related information corresponding to the batch program identifier from a database server; Call the corresponding target batch program through the microservice registration center server according to the batch program related information; When a new batch program is added, a new batch program identifier is added to the batch startup script; and the new batch program related information corresponding to the new batch program identifier is added to the database server through the microservice calling program.

2. The batch program processing method according to claim 1, characterized in that: The batch program related information includes: file directory, file format, user name and password.

3. The batch program processing method according to claim 1, characterized in that: The microservice registration center server includes: ZooKeeper.

4. The batch program processing method according to claim 1, characterized in that: The batch program is identified as monthreport.

5. A batch program processing device based on microservice architecture, characterized in that: Applied to batch start servers, the batch program processing device comprises: The running unit is used to run the batch startup script according to the batch running time configured by the batch scheduling server; A program initiation unit is used to initiate a microservice invocation program by running the batch initiation script, and send a batch program identifier to the microservice invocation program, wherein the batch program identifier is pre-configured by the batch scheduling server; the batch initiation server configures a batch initiation script and a batch program identifier, and the batch initiation script is used for different batch programs, and different batch programs correspond to different batch program identifiers in the batch initiation script; the batch initiation script includes code information and a batch program identifier, and the corresponding target batch program is determined based on the batch program identifier; An information acquisition unit, configured to call up a program through the microservice and acquire batch program related information corresponding to the batch program identifier from a database server; A program calling unit, configured to call the corresponding target batch program through the microservice registration center server according to the batch program related information; The identification adding unit is used to add a new batch program identification to the batch startup script when a new batch program is added; the information adding unit is used to add the new batch program related information corresponding to the new batch program identification in the database server through the microservice calling program.

6. A batch program processing system based on microservice architecture, characterized in that: include: A batch scheduling server, a batch startup server, a database server, a microservice registration center server and at least one microservice server, each of which stores a batch program; The batch scheduling server is connected to the batch startup server, and the batch scheduling server is used to configure batch running time, batch startup scripts and batch program identifiers in the batch startup server; The batch startup server is connected to the database server and the microservice registration center server respectively, and the batch startup server includes the batch program processing device according to claim 5; the batch startup script includes code information and a batch program identifier, and the corresponding target batch program is determined based on the batch program identifier; The database server is used to store the corresponding relationship between the batch program identifier and the batch program related information; The microservice registration center server is connected to each of the microservice servers and is used to call the corresponding target batch program according to the batch program related information sent by the batch startup server.

7. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the batch program processing method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the batch program processing method according to any one of claims 1 to 4.

Images