Task monitoring methods, devices, database tools, and electronic equipment

Abstract

This application provides a task monitoring method, apparatus, database tool, and electronic device. The method is applied to a database tool and includes: responding to a task monitoring instruction, calling an execution interface through a first target program running in the database tool to obtain queue status management information in a queue server; the task monitoring instruction is sent by a distributed task scheduling platform; based on the queue status management information, obtaining abnormal queue information in all queues; based on the abnormal queue information, determining a corresponding self-healing strategy through a second target program running in the database tool, and performing self-healing processing on the abnormal queues according to the self-healing strategy. This application can receive monitoring instructions from a distributed task scheduling platform, obtain queue status management information in a queue server through a first target program, and identify abnormal queues. The second target program performs self-healing processing on the abnormal queues, enabling timely recovery when queues are abnormal.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a task monitoring method, apparatus, database tool, and electronic device. Background Technology

[0002] Real-time data integration and replication software tools (Oracle Goldengate, OGG) offer advantages such as high database compatibility, low resource consumption, and simple configuration, giving them a significant edge over other data integration tools. Therefore, OGG is used to build synchronization queues when connecting the real-time bus to the database. However, OGG lacks monitoring capabilities when building synchronization queues, leading to difficulties in timely recovery when queue anomalies occur. Summary of the Invention

[0003] This application provides a task monitoring method, apparatus, database tool, and electronic device to at least solve the above-mentioned technical problems existing in the prior art.

[0004] A first aspect of this application provides a task monitoring method, the method being applied to a database tool; the method includes:

[0005] In response to a task monitoring command, the execution interface is invoked by the first target program running in the database tool to obtain the status management information of the queues in the queue server. The execution interface encapsulates control commands, which are used to obtain the status management information, including: status information of all queues or status information of the target queue, report information of the target queue's running status, or start or stop information of the target queue. The task monitoring command is sent by a distributed task scheduling platform. The queue server includes multiple queues, which are used to synchronize data between the source database and the target database. The queues include source queues and target queues.

[0006] Based on the status management information of the queues, obtain the abnormal queue information in all queues;

[0007] Based on the abnormal queue information, a corresponding self-healing strategy is determined by a second target program running in the database tool, and the abnormal queue is self-healed according to the self-healing strategy; the second target program has a self-healing strategy.

[0008] In one possible implementation, obtaining abnormal queue information from all queues based on the queue's state management information includes:

[0009] Based on the aforementioned status management information, identify the abnormal queues among all queues;

[0010] Obtain the log information of the exception queue, and retrieve the corresponding exception code from the log information as the exception queue information.

[0011] In one possible implementation, the step of determining the abnormal queues among all queues based on state management information includes at least one of the following:

[0012] When the status information of any queue in all queues or the status information of the target queue indicates an anomaly, the queue with the anomaly status information is determined to be an abnormal queue.

[0013] When the report information on the operating status of the target queue indicates an anomaly, the target queue is determined to be an abnormal queue.

[0014] When the start or stop information of the target queue indicates an abnormal start or stop, the target queue is determined to be an abnormal queue.

[0015] In one possible implementation, determining the abnormal queues among all queues based on the state management information includes:

[0016] When the existence of an abnormal queue is determined based on the status management information, the database tool is restarted;

[0017] Obtain the queue status management information in the queue server of the database tool after restart, and identify the abnormal queues among all queues based on this status management information.

[0018] In one possible implementation, obtaining the log information of the exception queue and retrieving the corresponding exception code from the log information as the exception queue information includes:

[0019] In response to the exception type represented by the log information of the exception queue being a source table structure change causing the target queue to be interrupted, the corresponding first exception code is obtained;

[0020] The log information of the exception queue indicates that the exception type is caused by the source character set being different from the target character set, resulting in the interruption of the target queue and the acquisition of the corresponding second exception code.

[0021] In response to the exception type represented by the log information of the exception queue being the full-column log of the source exception table being abnormally closed, causing the target queue to be interrupted, the corresponding third exception code is obtained;

[0022] In response to the exception type represented by the log information in the exception queue being source process interruption, the corresponding fourth exception code is obtained.

[0023] In one possible implementation, determining the corresponding self-healing strategy based on the anomaly queue information using a second target program running in a database tool includes:

[0024] The self-healing strategy corresponding to the first abnormal code is: obtain the source table structure and reload the table structure configuration on the target side;

[0025] The self-healing strategy corresponding to the determined second exception code is: to escape the characters in the source character set and then write them to the target end;

[0026] The self-healing strategy corresponding to the determined third exception code is as follows: block the source exception table, restart the target end so that the target end queue restores other tables except the exception table, the target end determines the queue name corresponding to the source exception table, and reloads the source file of the source exception table in the database;

[0027] The self-healing strategy corresponding to the determined fourth exception code is to restart the exception queue.

[0028] In one possible implementation, in response to a change instruction in the task triggering frequency, a task monitoring instruction adjusted by the distributed task scheduling platform according to the change in the task triggering frequency is received.

[0029] A second aspect of this application provides a task monitoring device, configured within a database tool, the device comprising:

[0030] The first target application is used to respond to task monitoring instructions, call the execution interface, and obtain the status management information of the queues in the queue server; it is also used to obtain abnormal queue information in all queues based on the status management information of the queues.

[0031] The execution interface encapsulates control commands, which are used to obtain the status management information, including: status information of all queues or status information of the target queue, report information of the target queue's running status, or start or stop information of the target queue; the task monitoring instruction is sent by the distributed task scheduling platform; the queue server includes multiple queues, which are used to synchronize data between the source database and the target database, and the queues include source queues and target queues;

[0032] The second target application is used to determine the corresponding self-healing strategy based on the abnormal queue information, and to perform self-healing processing on the abnormal queue according to the self-healing strategy; the second target application is equipped with a self-healing strategy.

[0033] A third aspect of this application provides a database tool, including: the task monitoring device and queue server provided in the above embodiments;

[0034] The queue server includes multiple queues, which are used to synchronize data between the source database and the target database. The queues include source queues and target queues.

[0035] A fourth aspect of this application provides an electronic device comprising:

[0036] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0037] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0038] The task monitoring method, apparatus, database tool, and electronic device of this application include a database tool that can receive monitoring instructions from a distributed task scheduling platform, obtain the status management information of queues in a queue server, identify abnormal queues, and perform self-healing on abnormal queues according to a self-healing processing strategy, so that the queue can be recovered in a timely manner when an abnormality occurs.

[0039] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0040] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, wherein:

[0041] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0042] Figure 1 A flowchart illustrating the task monitoring method according to an embodiment of this application is shown;

[0043] Figure 2 A schematic diagram of the structure of the database tool according to an embodiment of this application is shown;

[0044] Figure 3 Another flowchart of the task monitoring method according to an embodiment of this application is shown;

[0045] Figure 4 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0046] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] The following description, in conjunction with the accompanying drawings, introduces a task monitoring method, apparatus, database tool, and electronic device provided in this application.

[0048] like Figure 1 As shown, this application provides a task monitoring method, which is applied to a database tool; the method includes:

[0049] S101, in response to the task monitoring command, the execution interface is invoked by the first target program running in the database tool to obtain the status management information of the queues in the queue server; the execution interface encapsulates control commands, which are used to obtain the status management information, including: status information of all queues or status information of the target queue, report information of the target queue's running status, or start or stop information of the target queue; the task monitoring command is sent by the distributed task scheduling platform; the queue server includes multiple queues, which are used to synchronize data between the source database and the target database, and the queues include source queues and target queues.

[0050] In this application, the database tool is the OGG tool, which is a tool for data replication and integration. OGG achieves real-time data synchronization by capturing data changes (such as insert, update, and delete operations) in the source database and then applying these changes to the target database. The OGG tool captures data changes from the source (the origin of the data changes, i.e., where the data is captured and read) and sends them to the target (the place where the data changes are applied, i.e., the destination where the data is replicated).

[0051] It should be noted that the OGG tool is a pure command-line tool, and current monitoring tools cannot monitor the OGG queue within the OGG tool. Therefore, to monitor the OGG queue, such as... Figure 2As shown, the OGG tool provided in this application includes an OGG service and an OGG queue server. The first target program and execution interface are located within the OGG service. The first target program can obtain task monitoring instructions sent by the distributed task scheduling platform and call the execution interface to obtain the status management information of the queues in the OGG queue server, thereby achieving monitoring of the OGG queues. This application pre-encapsulates control commands into the execution interface. These control commands include obtaining status information of all queues in the OGG queue server or the status information of the target queue, a report of the target queue's running status, or start / stop information of the target queue.

[0052] In this application, the OGG service can define a command content using a string array. For example, the string array is {"info all"}, and the command content corresponding to this string array is "View all queue statuses". In this application, all the command content required for monitoring is encapsulated into the execution interface in the above manner. After receiving the task monitoring instruction, the first target program calls the execution interface to execute the control command and obtain the return result.

[0053] For example, in the OGG service, control commands (info all / get all queue status information, viewreport / get target queue running status report information, start / stop / get target queue start or stop information) are encapsulated as execution interfaces in a pipeline manner. These execution interfaces can also analyze the execution results of the control commands to obtain queue status management information in the queue server; this status management information is data that the first target program can parse. The first target program obtains the queue status management information in the queue server through the execution interface.

[0054] The OGG queue server comprises multiple OGG queues. These queues are used for real-time synchronization of data between the source database (referred to as the source) and the target database (referred to as the target). The queues include source queues and target queues. The source queue stores data captured from the source database (tables, character sets, table structures, transaction changes, etc.). The target queue stores data transmitted from the source database for use by the target database. If an interruption or other anomaly occurs during data transmission from the source to the target, the queue is considered an abnormal queue.

[0055] This application involves issuing task monitoring commands through a task monitoring executor on a distributed task scheduling platform. Users can configure these commands according to their specific needs. For example, the distributed task scheduling platform can be set to perform scheduled status queries; upon reaching the scheduled time, the platform will send a task monitoring command to the database tool.

[0056] S102, Based on the status management information of the queues, determine the abnormal queue information in all queues.

[0057] In this application, if the queue's status management information displays ABENDED, it indicates that the queue has encountered an anomaly. All queues with status management information displaying ABENDED are selected as abnormal queues. The abnormal queue information is then determined through these abnormal queues.

[0058] S103, based on the abnormal queue information, a corresponding self-healing strategy is determined by a second target program running in the database tool, and the abnormal queue is self-healed according to the self-healing strategy; the second target program has a self-healing strategy.

[0059] In this application, the OGG service also includes a second target program, which executes a pre-packaged self-healing strategy on the exception queue. After determining the exception queue information, the cause of the queue exception can be identified. The second target program contains a self-healing strategy, which is selected based on the cause of the exception. Executing the self-healing strategy completes the self-healing process for the exception queue, preventing queue interruptions due to exceptions, or repairing already interrupted queues, allowing the queue to resume operation promptly.

[0060] The task monitoring method provided in this application utilizes a distributed task scheduling platform to provide scheduled task monitoring instructions. Upon receiving the task monitoring instruction, the first target program of the OGG tool calls the execution interface. The execution interface executes its encapsulated control commands, obtains the queue status management information from the queue server, and determines whether an abnormal queue exists in the OGG queue. If so, a second target program performs self-healing processing on the abnormal queue. In this application, the first target program of the OGG tool can receive monitoring instructions from the distributed task scheduling platform, call the execution interface to obtain the queue status management information from the queue server, and identify abnormal queues. The second target program matches the abnormal queue with a self-healing strategy and uses the corresponding strategy to self-heal the abnormal queue, enabling timely recovery from queue anomalies.

[0061] In some embodiments, S102 above: Based on the state management information of the queue, obtain abnormal queue information in all queues, including:

[0062] Based on the aforementioned status management information, identify the abnormal queues among all queues;

[0063] Obtain the log information of the exception queue, and retrieve the corresponding exception code from the log information as the exception queue information.

[0064] In this application, when the status management information of the queue in the aforementioned queue server indicates an anomaly, the corresponding queue is identified as an abnormal queue. After identifying the abnormal queue, the log information of the abnormal queue is obtained. The log information records an exception code. This application allows filtering to obtain the exception code from the log information.

[0065] Based on state management information, the above identifies abnormal queues in all queues, including at least one of the following:

[0066] When the status information of any queue in all queues or the status information of the target queue indicates an anomaly, the queue with the anomaly status information is determined to be an abnormal queue.

[0067] When the report information on the operating status of the target queue indicates an anomaly, the target queue is determined to be an abnormal queue.

[0068] When the start or stop information of the target queue indicates an abnormal start or stop, the target queue is determined to be an abnormal queue.

[0069] If the status information of any queue or the status information of the target queue in this application indicates an anomaly, or if the report information of the target queue's running status indicates an anomaly, or if the start or stop information of the target queue indicates an anomaly in its start or stop, the status management information will display ABENDED. At this time, the corresponding abnormal queue will be identified.

[0070] In some embodiments, when it is determined that there is an abnormal queue based on the status management information, the database tool can be restarted to obtain the status management information of the queues in the queue server of the database tool after the restart, and the abnormal queues in all queues can be re-determined based on the status management information.

[0071] In this application, after determining that the status management information is ABENDED, the database tool is first restarted. Some exceptions that can be resolved by restarting can be handled in this step. If the status management information of the queues in the queue server is still ABENDED after restarting, the queues with ABENDED status are identified as abnormal queues.

[0072] In this application, if the cause of the abnormal queue is merely a temporary issue such as network packet loss or primary / backup switching of the source system, the queue abnormality can be resolved simply by restarting the database tool. There is no need for a secondary target program to perform self-healing processing on the abnormal queue, thus simplifying the exception handling process.

[0073] In some embodiments, obtaining the log information of the exception queue and retrieving the corresponding exception code from the log information as the exception queue information includes:

[0074] In response to the exception type represented by the log information of the exception queue being a source table structure change causing the target queue to be interrupted, the corresponding first exception code is obtained;

[0075] The log information of the exception queue indicates that the exception type is caused by the source character set being different from the target character set, resulting in the interruption of the target queue and the acquisition of the corresponding second exception code.

[0076] In response to the exception type represented by the log information of the exception queue being the full-column log of the source exception table being abnormally closed, causing the target queue to be interrupted, the corresponding third exception code is obtained;

[0077] In response to the exception type represented by the log information in the exception queue being source process interruption, the corresponding fourth exception code is obtained.

[0078] In this application, the exception queue can include multiple exception types, with different exception types corresponding to different exception codes. This allows for the determination of appropriate self-healing strategies for the exception queue in subsequent steps, based on the different exception codes.

[0079] For example, changes to the source table structure, such as modifying column names, deleting fields, or increasing field lengths, can cause the target OGG queue to become inconsistent with the source database if the real-time OGG tool is out of sync with it. This results in an abnormal interruption of the target queue (the status management information of the source and / or target queues may indicate an anomaly, and the queue logs may record information indicating the anomaly type). If the anomaly type is a target queue interruption caused by a change in the source table structure, the corresponding exception code is OGG-01163, which is the first exception code mentioned above.

[0080] If the source character set differs from the target character set, causing an abnormal interruption of the target queue (the status management information of the source queue and / or the target queue may indicate an exception, and the queue's log information may record relevant information representing the exception type). If the exception type is an abnormal interruption of the target queue due to a difference in the source and target character sets, the corresponding exception code is OGG-03533, which is the second exception code mentioned above.

[0081] If the full-column log of the source table is abnormally closed due to table reconstruction or other reasons, resulting in incomplete data fields in the database log, a mapping conflict will occur during synchronization to the target end, causing the target queue to be abnormally interrupted (in which case the status management information of the source queue and / or the target queue may indicate an anomaly, and the queue's log information may record relevant information indicating the anomaly type). If the anomaly type is that the abnormal closure of the full-column log of the source table caused the target queue to be interrupted, the corresponding anomaly code is OGG-00662, which is the third anomaly code mentioned above.

[0082] A source process interruption, meaning a data transmission interruption from the source to the target, causes an abnormal interruption of the target queue (in which case the status management information of the source queue and / or the target queue may indicate an anomaly, and the queue's log information may record relevant information indicating the anomaly type). If the anomaly type is source process interruption causing target queue interruption, the corresponding anomaly code is OGG-00662, which is the fourth anomaly code mentioned above.

[0083] In some embodiments, the self-healing process of the exception queue by a second target program running in the database tool includes:

[0084] Based on the exception code in the log information of the exception queue, obtain the self-healing processing strategy corresponding to the exception code in the log information, and use the self-healing processing strategy to perform self-healing processing on the exception queue.

[0085] In this application, the second target program stores exception codes and corresponding self-healing strategies. Therefore, after determining the exception code in the log information, the corresponding self-healing strategy can be obtained from the self-healing strategy based on the exception code, thereby using the self-healing strategy to perform self-healing processing on the exception queue.

[0086] In some embodiments, based on the anomaly queue information, a corresponding self-healing strategy is determined by a second target program running in a database tool, including:

[0087] The self-healing strategy corresponding to the first abnormal code is: obtain the source table structure and reload the table structure configuration on the target side;

[0088] The self-healing strategy corresponding to the determined second exception code is: to escape the characters in the source character set and then write them to the target end;

[0089] The self-healing strategy corresponding to the determined third exception code is as follows: block the source exception table, restart the target end so that the target end queue restores other tables except the exception table, the target end determines the queue name corresponding to the source exception table, and reloads the source file of the source exception table in the database;

[0090] The self-healing strategy corresponding to the determined fourth exception code is to restart the exception queue.

[0091] For the first exception code OGG-01163, the first step is to add a table structure reload configuration to the target's configuration in the OGG tool. Then, based on the original configuration information of the target, the corresponding source process name is located, and the table structure file of the synchronized table is retrieved again according to the synchronization information configured on the source. Because the target has already configured table structure reload, the corresponding self-healing strategy for exception code OGG-01163 is to restart the target queue. The exception queue can then obtain the latest table structure information, and self-healing is complete.

[0092] Regarding the second exception code OGG-03533, the target end detected a character set conflict exception, indicating that the synchronized data contained characters that the target end could not parse. The corresponding self-healing strategy for exception code OGG-03533 writes the character escaping configuration REPLACEBADCHAR NULL to the target end's queue file. This configuration means that the abnormal character will be converted to null, and the target end process will be automatically restarted, completing the self-healing process.

[0093] For the third exception code OGG-01296, which indicates that a table being synchronized in the source database has undergone reconstruction or other operations, the source database does not actively enable full-column logging for the corresponding table. This results in incomplete DML operation fields recorded in the logs, and the data sent to the OGG target database cannot be processed due to these incomplete fields. In this case, the self-healing strategy for exception code OGG-01296 will first disable synchronization of this table and restart the queue to restore synchronization of other tables. Simultaneously, the target database will look up the corresponding source queue name and, using the database address configured in the source queue's login file, execute the command to re-add the full-column log for table1. After this command is executed successfully, the target database will unblock the exception table and restart to restore normal synchronization of all tables, completing the self-healing process.

[0094] Regarding the fourth exception code, OGG-00662, the production synchronization queue for the Oracle cloud database experiences an interruption every six hours. While a restart resolves the issue, it often results in useless alerts. To address this, the OGG service has added a proactive restart function to its queue monitoring capabilities. Specifically, when an OGG queue exception is detected, the self-healing strategy for exception code OGG-00662 is to execute a restart command and then re-acquire the queue status after 5 minutes.

[0095] The task monitoring method provided in this application performs self-healing processing on the abnormal queue. If the status is still abnormal after self-healing, subsequent self-healing and alarm measures are finally executed.

[0096] In some embodiments, in response to a change instruction in the task triggering frequency, a task monitoring instruction adjusted by the distributed task scheduling platform according to the change in the task triggering frequency is received.

[0097] In this application, users can set the trigger frequency of monitoring tasks on the distributed task scheduling platform as needed. For example, if the current task trigger frequency is once every 5 hours, it can be reset to once every 2 hours.

[0098] In addition, this application can also encapsulate a new self-healing strategy into a second target program to achieve self-healing processing of newly emerging exception queues.

[0099] like Figure 3 As shown in the illustration, in a specific embodiment, the task monitoring method provided in this application firstly involves the first target program in the OGG service obtaining the queue status of the OGG queue in the OGG queue server according to the task monitoring instruction sent by the distributed task scheduling platform. If the queue status is ABENDED, the OGG tool can be restarted first. After restarting the OGG tool, the status management information of the queue in the queue server is obtained, and the status management information of the queue is re-determined to be ABENDED. If it is, the abnormal queue is self-healed using the self-healing processing strategy in the second target program. If the queue status of the abnormal queue is still ABENDED after self-healing, an abnormal alarm is sent.

[0100] The task monitoring method provided in this application can monitor the status of OGG queues based on a distributed task scheduling platform, and monitor the status management information of all current OGG queues. It enables self-healing of abnormal OGG queues by matching self-healing handling strategies to these queues, allowing for timely recovery from queue interruptions and significantly reducing the operational burden on maintenance personnel.

[0101] OGG tools have multiple versions, each corresponding to a different source database. The task monitoring method provided in this application can be applied to any version of OGG tools.

[0102] This application provides a task monitoring device, namely Figure 2 The OGG service shown is set up in a database tool, and the device includes:

[0103] The first target application is used to respond to task monitoring instructions, call the execution interface, and obtain the status management information of the queues in the queue server; it is also used to obtain abnormal queue information in all queues based on the status management information of the queues.

[0104] The execution interface encapsulates control commands, which are used to obtain the status management information, including: status information of all queues or status information of the target queue, report information of the target queue's running status, or start or stop information of the target queue; the task monitoring instruction is sent by the distributed task scheduling platform; the queue server includes multiple queues, which are used to synchronize data between the source database and the target database, and the queues include source queues and target queues;

[0105] The second target application is used to determine the corresponding self-healing strategy based on the abnormal queue information, and to perform self-healing processing on the abnormal queue according to the self-healing strategy; the second target application is equipped with a self-healing strategy.

[0106] The task monitoring device provided in this application is installed within a database tool. The device includes a first target application and a second target application. The first target application, upon receiving a task monitoring instruction, can invoke an execution interface to obtain queue status management information from a queue server; it is also used to obtain abnormal queue information from all queues based on the queue status management information. The second target application is used to determine a corresponding self-healing strategy based on the abnormal queue information and to perform self-healing processing on the abnormal queues according to the self-healing strategy; the second target application includes a self-healing strategy.

[0107] The task monitoring device provided in this application can receive monitoring instructions from a distributed task scheduling platform, obtain the status management information of queues in the queue server, and identify abnormal queues. It then performs self-healing on the abnormal queues according to a self-healing handling strategy, enabling timely recovery when queues experience anomalies.

[0108] Continue as Figure 2 As shown, this application provides a database tool, including: the task monitoring device and queue server provided in the above embodiments;

[0109] The queue server includes multiple queues, which are used to synchronize data between the source database and the target database. The queues include source queues and target queues.

[0110] It should be noted that the task monitoring device in this application embodiment is similar in principle to the aforementioned task monitoring method in solving the problem. Therefore, the implementation process, implementation principle, and beneficial effects of the task monitoring device can be found in the description of the implementation process, implementation principle, and beneficial effects of the aforementioned method. Repeated descriptions will not be repeated.

[0111] This application provides an electronic device, including:

[0112] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0113] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in any of the above embodiments.

[0114] This application provides a non-transitory computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the methods described in any of the above embodiments.

[0115] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0116] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0117] like Figure 4 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0118] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0119] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the task monitoring method. For example, in some embodiments, the task monitoring method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the task monitoring method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the task monitoring method by any other suitable means (e.g., by means of firmware).

[0120] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0122] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0124] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0125] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0126] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0127] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A task monitoring method, characterized in that, The method is applied to database tools; the method includes: In response to a task monitoring command, the execution interface is invoked by the first target program running in the database tool to obtain the status management information of the queues in the queue server. The execution interface encapsulates control commands, which are used to obtain the status management information, including: status information of all queues or status information of the target queue, report information of the target queue's running status, or start or stop information of the target queue. The task monitoring command is sent by a distributed task scheduling platform. The queue server includes multiple queues, which are used to synchronize data between the source database and the target database. The queues include source queues and target queues. Based on the status management information of the queues, obtain the abnormal queue information in all queues; Based on the abnormal queue information, a corresponding self-healing strategy is determined by a second target program running in the database tool, and the abnormal queue is self-healed according to the self-healing strategy; the second target program has a self-healing strategy. The step of obtaining abnormal queue information in all queues based on the status management information of the queues includes: determining abnormal queues in all queues based on the status management information; obtaining log information of the abnormal queues; and obtaining the corresponding abnormal code from the log information as the abnormal queue information. The step of obtaining the log information of the exception queue and retrieving the corresponding exception code from the log information as the exception queue information includes: obtaining a first exception code in response to the exception type represented by the log information of the exception queue being a source table structure change causing the target queue to be interrupted; obtaining a second exception code in response to the exception type represented by the log information of the exception queue being a source character set different from the target character set causing the target queue to be interrupted; obtaining a third exception code in response to the exception type represented by the log information of the exception queue being a full column log abnormal closure of the source exception table causing the target queue to be interrupted; and obtaining a fourth exception code in response to the exception type represented by the log information of the exception queue being a source process interruption. The step of determining the corresponding self-healing strategy based on the exception queue information by using a second target program running in the database tool includes: the self-healing strategy corresponding to the first exception code is: obtaining the source table structure and reloading the table structure on the target; the self-healing strategy corresponding to the second exception code is: escaping the characters in the source character set and writing them to the target; the self-healing strategy corresponding to the third exception code is: masking the source exception table, restarting the target so that the target queue recovers all tables except the exception table, the target determines the queue name corresponding to the source exception table, and reloads the source file of the source exception table in the database; and the self-healing strategy corresponding to the fourth exception code is: restarting the exception queue.

2. The method according to claim 1, characterized in that, The method of identifying abnormal queues among all queues based on state management information includes at least one of the following: When the status information of any queue in all queues or the status information of the target queue indicates an anomaly, the queue with the anomaly status information is determined to be an abnormal queue. When the report information on the operating status of the target queue indicates an anomaly, the target queue is determined to be an abnormal queue. When the start or stop information of the target queue indicates an abnormal start or stop, the target queue is determined to be an abnormal queue.

3. The method according to claim 1 or 2, characterized in that, The step of identifying abnormal queues among all queues based on the state management information includes: When the existence of an abnormal queue is determined based on the status management information, the database tool is restarted; Obtain the queue status management information in the queue server of the database tool after restart, and identify the abnormal queues among all queues based on this status management information.

4. The method according to claim 1, characterized in that, In response to changes in task trigger frequency, it receives task monitoring instructions from the distributed task scheduling platform to adjust according to changes in task trigger frequency.

5. A task monitoring device, characterized in that, The device, which is configured within a database tool, includes: The first target application is used to respond to task monitoring instructions, call the execution interface, and obtain the status management information of the queues in the queue server; it is also used to obtain abnormal queue information in all queues based on the status management information of the queues. The execution interface encapsulates control commands, which are used to obtain the status management information, including: status information of all queues or status information of the target queue, report information of the target queue's running status, or start or stop information of the target queue; the task monitoring instruction is sent by the distributed task scheduling platform; the queue server includes multiple queues, which are used to synchronize data between the source database and the target database, and the queues include source queues and target queues; The second target application is used to determine the corresponding self-healing strategy based on the abnormal queue information, and to perform self-healing processing on the abnormal queue according to the self-healing strategy; the second target application is equipped with a self-healing strategy. The first target application is further configured to determine the abnormal queues in all queues based on the state management information; obtain the log information of the abnormal queues; and obtain the corresponding abnormal code from the log information as the abnormal queue information. The first target application is further configured to: in response to the log information of the exception queue indicating that the exception type is a source-side table structure change causing the target-side queue to be interrupted, obtain the corresponding first exception code; in response to the log information of the exception queue indicating that the exception type is a source-side character set different from the target-side character set causing the target-side queue to be interrupted, obtain the corresponding second exception code; in response to the log information of the exception queue indicating that the exception type is a source-side exception table's full-column log abnormal closure causing the target-side queue to be interrupted, obtain the corresponding third exception code; and in response to the log information of the exception queue indicating that the exception type is a source-side process interruption, obtain the corresponding fourth exception code. The second target application further uses the following self-healing strategy to determine the first exception code: obtain the source table structure and reload the table structure on the target end; the determined self-healing strategy to determine the second exception code is: escape the characters in the source character set and write them to the target end; the determined self-healing strategy to determine the third exception code is: block the source exception table, restart the target end so that the target end queue restores all tables except the exception table, the target end determines the queue name corresponding to the source exception table, and reloads the source file of the source exception table in the database; the determined self-healing strategy to determine the fourth exception code is: perform a restart operation on the exception queue.

6. A database tool, characterized in that, include: The task monitoring device and queue server as described in claim 5; The queue server includes multiple queues, which are used to synchronize data between the source database and the target database. The queues include source queues and target queues.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

Images