Data Synchronization Method, Device, Equipment and Storage Medium

Abstract

The present invention relates to a data synchronization method, apparatus, device and storage medium based on the monitoring of the status of processing nodes. The method includes: receiving a synchronization request from a data requester; in response to the data synchronization request, detecting whether the data to be synchronized has been synchronized to the CSS temporary table; if it has been synchronized to the CSS temporary table, detecting whether the data to be synchronized has completed data extraction; if the data extraction has been completed, detecting whether the Saturn file transfer task has been successfully scheduled; if the scheduling is successful, detecting whether the data has been unloaded; if the data has been unloaded, detecting whether the data has been synchronized to the CHANNEL temporary table; if it has been synchronized to the CHANNEL temporary table, detecting whether the data has been synchronized to the UM system; if it has been synchronized to the UM system, the data synchronization is completed. By introducing the status monitoring of processing nodes to assist data synchronization, the real-time performance of data synchronization is ensured, the test efficiency is improved, and the technical problems of poor real-time performance of existing data synchronization and low test efficiency are solved.

Description

Technical Field

[0001] The present application relates to the field of monitoring, and in particular to a node monitoring and correction technology through software, and specifically discloses a data synchronization method, device, equipment and storage medium based on processing node status monitoring. Background Art

[0002] Due to environmental barriers and scattered nodes in the current test environment, the created account data cannot be automatically synchronized to other required application systems. In particular, the wireless network has the characteristics of limited bandwidth resources, unstable link status, and large communication delay. As the number of messages sent increases, the load of message diffusion also increases; and the data synchronization link is long and involves many applications. The existing process requires manual search for the person in charge of each synchronization node application to promote data synchronization, which greatly affects data real-time performance and test efficiency.

[0003] Therefore, how to solve the existing technical problems of poor real-time data synchronization and low testing efficiency has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present invention is to provide a data synchronization method, device, equipment and storage medium based on processing node status monitoring, aiming to solve the technical problems of poor real-time performance and low testing efficiency of existing data synchronization.

[0005] To achieve the above-mentioned purpose, the present invention proposes a data synchronization method based on processing node status monitoring, wherein the method includes: receiving a synchronization request from a data demander; responding to the data synchronization request, detecting whether the data to be synchronized has been synchronized to a CSS temporary table; if it has been synchronized to the CSS temporary table, detecting whether the data to be synchronized has completed data extraction; if the data extraction has been completed, detecting whether the Saturn file transfer task is successfully scheduled; if the scheduling is successful, detecting whether the data has been unloaded; if it has been unloaded, detecting whether the data has been synchronized to a CHANNEL temporary table; if it has been synchronized to the CHANNEL temporary table, detecting whether the data has been synchronized to a UM system; if it has been synchronized to the UM system, completing data synchronization.

[0006] In one of the embodiments, after the step of detecting whether the data to be synchronized has been synchronized to the CSS temporary table, the method further includes: if it has not been synchronized to the CSS temporary table, scheduling a CSS synchronization trigger to synchronize the data to the CSS temporary table; if it has not been synchronized to the CSS temporary table after a first preset number of consecutive schedulings, generating a first warning message, and sending the first warning message to relevant personnel.

[0007] In one embodiment, after the step of detecting whether the data to be synchronized has completed data extraction, the method further includes: if the data extraction is not completed, logging in to the DS platform to schedule the corresponding data extraction task; if the data extraction is still not completed after continuously scheduling for a second preset number of times, generating a second warning message and sending the second warning message to the relevant personnel.

[0008] In one embodiment, after the step of detecting whether the Saturn file transfer task is scheduled successfully, the method further includes: if the scheduling is not successful, logging in to the Saturn platform to schedule the corresponding file transfer task; if the Saturn file transfer task is still not completed after continuously scheduling for a third preset number of times, generating a third warning message and sending the third warning message to the relevant personnel.

[0009] In one embodiment, after the step of detecting whether the data has been unloaded, the method further includes: if the unloading is not completed, logging in to the ODS platform to schedule the corresponding unloading task; if the unloading is still not successful after continuously scheduling for a fourth preset number of times, generating a fourth warning message and sending the fourth warning message to the relevant personnel.

[0010] In one embodiment, after the step of detecting whether the data has been synchronized to the CHANNEL temporary table, the method further includes: if the data has not been synchronized to the CHANNEL temporary table, scheduling the CHANNEL synchronization trigger to synchronize the data to the CHANNEL temporary table; if the data has still not been synchronized to the CHANNEL temporary table after continuously scheduling for a fifth preset number of times, generating a fifth warning message and sending the fifth warning message to the relevant personnel.

[0011] In one embodiment, after the step of detecting whether the data has been synchronized to the UM system, the method further includes: if the data has not been synchronized to the UM system, logging in to the OGG platform to schedule the corresponding synchronization task; if the data has still not been synchronized to the UM system after continuously scheduling for a sixth preset number of times, generating a sixth warning message and sending the sixth warning message to the relevant personnel.

[0012] To achieve the above object, the present invention proposes a data synchronization device, wherein the device includes: a receiving module, configured to receive a synchronization request from a data requester; a processing node monitoring module, configured to respond to the data synchronization request, detect whether the data to be synchronized has been synchronized to the CSS temporary table, if it has been synchronized to the CSS temporary table, detect whether the data to be synchronized has completed data extraction, if the data extraction has been completed, detect whether the Saturn file transfer task is scheduled successfully, if the scheduling is successful, detect whether the data has been unloaded, if the unloading has been completed, detect whether the data has been synchronized to the CHANNEL temporary table, if it has been synchronized to the CHANNEL temporary table, detect whether the data has been synchronized to the UM system.

[0013] To achieve the above object, the present invention provides an electronic device, wherein the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the data synchronization method based on processing node status monitoring in the above method embodiments.

[0014] To achieve the above object, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the data synchronization method based on processing node status monitoring in the above method embodiments.

[0015] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0016] In the embodiments of the present invention, a synchronization request from a data requester is received; in response to the data synchronization request, it is detected whether the data to be synchronized has been synchronized to the CSS temporary table; if it has been synchronized to the CSS temporary table, it is detected whether the data to be synchronized has completed data extraction; if the data extraction has been completed, it is detected whether the Saturn file transfer task has been successfully scheduled; if the scheduling is successful, it is detected whether the data has been unloaded; if the data has been unloaded, it is detected whether the data has been synchronized to the CHANNEL temporary table; if it has been synchronized to the CHANNEL temporary table, it is detected whether the data has been synchronized to the UM system; if it has been synchronized to the UM system, the data synchronization is completed. By introducing the status monitoring of processing nodes to assist data synchronization, the real-time performance of data synchronization is ensured, the testing efficiency is improved, and the technical problems of poor real-time performance of existing data synchronization and low testing efficiency are solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0018] Figure 1 is a schematic diagram of an application environment of the data synchronization method based on processing node status monitoring provided in Embodiment 1 of the present invention;

[0019] Figure 2 is a flowchart of the data synchronization method based on processing node status monitoring provided in Embodiment 2 of the present invention;

[0020] Figure 3It is a flowchart of the data synchronization method based on the monitoring of the processing node status provided in the third embodiment of the present invention;

[0021] Figure 4 It is a flowchart of the data synchronization method based on the monitoring of the processing node status provided in the fourth embodiment of the present invention;

[0022] Figure 5 It is a flowchart of the data synchronization method based on the monitoring of the processing node status provided in the fifth embodiment of the present invention;

[0023] Figure 6 It is a flowchart of the data synchronization method based on the monitoring of the processing node status provided in the sixth embodiment of the present invention;

[0024] Figure 7 It is a flowchart of the data synchronization method based on the monitoring of the processing node status provided in the seventh embodiment of the present invention;

[0025] Figure 8 It is a flowchart of the data synchronization method based on the monitoring of the processing node status provided in the eighth embodiment of the present invention;

[0026] Figure 9 It is a schematic structural diagram of the data synchronization device provided in the ninth embodiment of the present invention;

[0027] Figure 10 It is a schematic structural diagram of a computer device provided in the tenth embodiment of the present invention. Detailed implementation manners

[0028] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0029] It should be understood that when used in the specification and appended claims of the present invention, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0030] It should also be understood that the term "and / or" used in the specification and appended claims of the present invention refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0031] As used in the specification of the present invention and the appended claims, the term "if" can be construed as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be construed as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" depending on the context.

[0032] In addition, in the description of the specification of the present invention and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0033] The reference to "one embodiment" or "some embodiments" or the like described in the specification of the present invention means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present invention. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0034] It should be understood that the magnitudes of the sequence numbers of the steps in the following embodiments do not mean the order of execution is prior or posterior, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0035] To illustrate the technical solution of the present invention, the following specific embodiments are used for illustration.

[0036] The data synchronization method based on processing node status monitoring provided in the first embodiment of the present invention can be applied in an application environment such as Figure 1 where the client communicates with the server. Among them, the client includes but is not limited to computer devices such as a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a personal digital assistant (PDA), etc. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0037] See Figure 2, is a schematic flowchart of the data synchronization method based on processing node status monitoring provided in the second embodiment of the present invention. The data synchronization method based on processing node status monitoring can be applied to Figure 1 the server in Figure 2 As shown, the data synchronization method based on processing node status monitoring may include the following steps:

[0038] Step S201: Receive a synchronization request from a data requester.

[0039] Generally, each business system has a corresponding trading platform, such as a bank's website, a securities trading software, etc. As Figure 1 described in the embodiment, the server of each business system is used to process various business data on the trading platform of the business system and can statistically store and store the data corresponding to each account according to the business data. The data requester can directly send a data synchronization to the server, request to directly obtain data from the server of the target business system, and the server receives the data synchronization request and sends the data to the data requester.

[0040] Of course, in actual applications, the server of the business system may also store data in a third-party database (such as a cloud database, etc.). In this case, the data requester can first send a data synchronization to the server, request to obtain data from the server, and the server receives the data synchronization request, retrieves data from the third-party database according to the request, and then sends the retrieved data to the data requester.

[0041] Step S202: In response to the data synchronization request, detect whether the data to be synchronized has been synchronized to the Common Search Space (CSS) temporary table.

[0042] Step S203: If it has been synchronized to the CSS temporary table, detect whether the data to be synchronized has completed data extraction.

[0043] Specifically, data extraction can be performed by listening, periodically, or waiting for a user instruction. Among them, the listening method: when new data is entered into the data source, the data extraction tool detects the generation of new data and extracts the data; the periodic method: the data extraction tool periodically extracts the newly generated data; the method of waiting for a user instruction: manually upload data by the user.

[0044] Step S204: If the data extraction has been completed, detect whether the Saturn file transfer task has been successfully scheduled.

[0045] Specifically, the scheduling platform is responsible for scheduling the audited data, and can monitor the data flow direction of the audited data. For the embodiments of the present invention, the scheduling platform can be implemented through the task scheduling system Saturn. Specifically, each business logic is encapsulated into a task, published to this common platform, and then the task and resource configuration, task control and management, task execution status monitoring, abnormal task display and alarm, etc. are carried out, replacing the traditional Linux Cron / Spring Batch Job method to achieve global unified configuration, unified monitoring, high task availability, and sharding concurrent processing.

[0046] Step S205, if the scheduling is successful, then detect whether the data has been unloaded.

[0047] Specifically, data unloading: The data provider (data providing party) exports and writes the data from the database of its own node to an external file according to the standard data exchange method, and uses a data transfer tool to push the data to the data requester.

[0048] Before the data requester uses the data unloading tool to unload the data in the database to the target data table, it usually needs to first formulate the requirements of the target data table to form a corresponding data unloading requirement file. The data unloading tool parses the data unloading requirement file of the data providing party to obtain the basic information of the target data table. Among them, the data unloading requirement file is used to record the basic information and related requirements of the target data table. The above basic information includes table name, data unloading date, data unloading method, multi-entity identifier, additional conditions, etc.

[0049] Step S206, if the data has been unloaded, then detect whether the data has been synchronized to the CHANNEL temporary table.

[0050] Specifically, through a certain mechanism, the data to be synchronized obtained from database A is sent to database B, and the data is loaded into database B through SQL specifications. The loading process can include: starting a transaction Txa; inserting the channel_id identifier of database A into the transaction; inserting business data into the transaction; clearing the channel_id identifier of database A; and committing the transaction Txa to database B.

[0051] Step S207, if the data has been synchronized to the CHANNEL temporary table, then detect whether the data has been synchronized to the UM system.

[0052] Specifically, for data integration, the most commonly used technical means are to use methods such as ETL, OGG, DBlink, WebServic, etc. to complete the extraction, transformation, and loading process of data from the business system to the data warehouse.

[0053] Step S208, if the data has been synchronized to the UM system, then complete the data synchronization.

[0054] Furthermore, in the embodiments of the present invention, the status monitoring software of the processing node can be used to monitor the status of the above-mentioned processing node, and the status monitoring software of the processing node is installed on the server side; the server side can be a terminal, for example, a mobile phone or a tablet computer, and the server side can also be a server, or a service cluster composed of multiple servers.

[0055] Specifically, the method for monitoring the status of the above-mentioned processing node includes:

[0056] Step 1, obtain the processing status information of the processing node.

[0057] Among them, the processing status information is used to represent the processing status of the processing node, including the normal status and the stagnant status. The stagnant status indicates that a processing error has occurred in the processing node. Among them, the processing status information can be directly generated by the processing node. For example, when the processing node is in the stagnant status, the processing node generates the processing status information representing the stagnant status and sends it to the status monitoring software. When the processing node is in the normal status, the processing node can generate the processing status information representing the normal status and send it to the server side installed with the status monitoring software.

[0058] Step 2, if the processing status information represents that the processing node is in the stagnant status, then generate a target control instruction according to the preset processing strategy.

[0059] Among them, the processing strategy is pre-configured. The processing strategy represents which operation the processing node needs to execute. Specifically, the processing strategy can include an exception skipping strategy and a delay retry strategy; the exception skipping strategy represents that the processing node can skip some data in the operation data, and then process the operation data after skipping again.

[0060] Step 3, send the target control instruction to the processing node; the target control instruction is used to control the processing node to execute the target data processing operation.

[0061] Among them, the target data processing operation refers to the processing operation of the processing node on the operation data based on the target control instruction.

[0062] Step 4, obtain the operation status information of the processing node.

[0063] Among them, the operation status information is used to represent the attempt result of the processing node for self-healing attempt, including operation success and operation failure; among them, the operation status information is similar to the above-mentioned processing status information, and it can be directly generated by the processing node. For example, when the processing node successfully executes the target data processing operation, the processing node generates the operation status information representing operation success and sends it to the status monitoring software. When the processing node fails to execute the target data processing operation, the processing node can generate the operation status information representing operation failure and send it to the status monitoring software.

[0064] Step 5, if the operation status information indicates that the target data processing operation fails, output a warning message according to the processing node;

[0065] Among them, if the operation status information is generated and sent by the processing node, after the status monitoring software receives the operation status information indicating successful operation, the subsequent steps can be executed; if the status monitoring software receives the operation status information indicating failed operation, the subsequent steps are not executed.

[0066] Among them, the warning message can be output in various ways, such as by phone, email, text message, etc. It can be understood that the data forms of the warning messages corresponding to different output methods are different, including voice, text, pictures and videos, etc. The warning message is mainly used to remind the management personnel to perform manual repair;

[0067] Among them, the output of the warning message is based on the processing node. Therefore, the management personnel can determine the processing node according to the warning message, that is, they can know which processing node needs to be manually repaired currently; specifically, the relevant information that can represent the identity of the processing node can be directly added to the warning message, such as the real-time node position of the processing node in the entire processing system at present, so that the management personnel can directly locate the processing node according to the real-time node position.

[0068] See Figure 3 , which is a schematic flowchart of the data synchronization method based on processing node status monitoring provided in Embodiment 3 of the present invention. After step S202, the method further includes:

[0069] Step S301: If the CSS temporary table is not synchronized, schedule the CSS synchronization trigger to synchronize the data to the CSS temporary table.

[0070] Step S302: If the CSS temporary table is still not synchronized after continuously scheduling the first preset number of times, generate a first warning message and send the first warning message to the relevant personnel.

[0071] Among them, the first preset number of times can be 3 - 5 times, and the first warning message is sent in the way of display interface or email sending or text message notification according to the predefined notification strategy.

[0072] See Figure 4 , which is a schematic flowchart of the data synchronization method based on processing node status monitoring provided in Embodiment 4 of the present invention. After step S203, the method further includes:

[0073] Step S401: If the data extraction is not completed, log in to the DS platform to schedule the corresponding data extraction task.

[0074] Step S402: If the data extraction is still not completed after continuously scheduling for the second preset number of times, generate a second warning message and send the second warning message to the relevant personnel.

[0075] Among them, the second preset number of times can be 3 - 5 times, and the second warning message is sent in a pre - defined notification strategy through a display interface, email, or SMS notification.

[0076] See Figure 5 , which is a schematic flowchart of the data synchronization method based on processing node status monitoring provided in the fifth embodiment of the present invention. After step S204, the method further includes:

[0077] Step S501: If the scheduling is not successful, log in to the Saturn platform to schedule the corresponding file transfer task.

[0078] Step S502: If the Saturn file transfer task is still not completed after continuously scheduling for the third preset number of times, generate a third warning message and send the third warning message to the relevant personnel.

[0079] Among them, the third preset number of times can be 3 - 5 times, and the third warning message is sent in a pre - defined notification strategy through a display interface, email, or SMS notification.

[0080] See Figure 6 , which is a schematic flowchart of the data synchronization method based on processing node status monitoring provided in the sixth embodiment of the present invention. After step S205, the method further includes:

[0081] Step S601: If the data unloading is not completed, log in to the ODS platform to schedule the corresponding data unloading task.

[0082] Among them, the synchronization mechanism can be controlled by formulating the priority of the ODS platform scheduling. When a corresponding task instance is generated or a task is being executed before the ODS platform scheduling, the table structure synchronization is postponed, a task instance dependency is established, and the table structure synchronization is executed after the task instance execution ends; when a corresponding task instance is generated during the ODS platform scheduling, the task instance is suspended and triggered to continue execution after the ODS platform scheduling ends; if no corresponding task instance is generated before and after the ODS platform scheduling process, the ODS executes normally.

[0083] Step S602: If the data unloading is still not successful after continuously scheduling for the fourth preset number of times, generate a fourth warning message and send the fourth warning message to the relevant personnel.

[0084] Among them, the fourth preset number of times can be 3 - 5 times, and the fourth warning message is sent in a pre - defined notification strategy through a display interface, email, or SMS notification.

[0085] See Figure 7 , which is a schematic flowchart of the data synchronization method based on processing node status monitoring provided in the seventh embodiment of the present invention. After step S206, the method further includes:

[0086] Step S701: If the data is not synchronized to the CHANNEL temporary table, schedule the CHANNEL synchronization trigger to synchronize the data to the CHANNEL temporary table.

[0087] Step S702: If the data is still not synchronized to the CHANNEL temporary table after continuously scheduling the fifth preset number of times, generate a fifth warning message and send the fifth warning message to the relevant personnel.

[0088] Among them, the fifth preset number of times can be 3 - 5 times, and the fifth warning message is sent in a way of display interface or email or SMS notification according to a predefined notification policy.

[0089] See Figure 8 , which is a schematic flowchart of the data synchronization method based on processing node status monitoring provided in the eighth embodiment of the present invention. After step S207, the method further includes:

[0090] Step S801: If the data is not synchronized to the UM system, log in to the OGG platform to schedule the corresponding synchronization task.

[0091] Step S802: If the data is still not synchronized to the UM system after continuously scheduling the sixth preset number of times, generate a sixth warning message and send the sixth warning message to the relevant personnel.

[0092] Among them, the sixth preset number of times can be 3 - 5 times, and the sixth warning message is sent in a way of display interface or email or SMS notification according to a predefined notification policy.

[0093] Corresponding to the data synchronization method based on processing node status monitoring in the above embodiments, Figure 9 shows a structural block diagram of the data synchronization device provided in the ninth embodiment of the present invention. The above data synchronization device is applied to the client. For the sake of convenience of description, only the parts related to the embodiments of the present invention are shown.

[0094] See Figure 9 , the data synchronization device 90 includes:

[0095] A receiving module 91, configured to receive a synchronization request from a data requester;

[0096] The processing node monitoring module 92 is used to respond to the data synchronization request, detect whether the data to be synchronized has been synchronized to the CSS temporary table. When it has been synchronized to the CSS temporary table, detect whether the data to be synchronized has completed data extraction. When the data extraction has been completed, detect whether the Saturn file transfer task has been successfully scheduled. When the scheduling is successful, detect whether the data has been unloaded. When the data has been unloaded, detect whether the data has been synchronized to the CHANNEL temporary table. When it has been synchronized to the CHANNEL temporary table, detect whether the data has been synchronized to the UM system.

[0097] The processing node monitoring module 92 is also used to schedule the CSS synchronization trigger to synchronize the data to the CSS temporary table when it has not been synchronized to the CSS temporary table; when it still has not been synchronized to the CSS temporary table after continuously scheduling the first preset number of times, generate a first warning message and send the first warning message to the relevant personnel.

[0098] The processing node monitoring module 92 is also used to log in to the DS platform to schedule the corresponding data extraction task when the data extraction has not been completed; when the data extraction still has not been completed after continuously scheduling the second preset number of times, generate a second warning message and send the second warning message to the relevant personnel.

[0099] The processing node monitoring module 92 is also used to log in to the Saturn platform to schedule the corresponding file transfer task when the scheduling is not successful; when the Saturn file transfer task still has not been completed after continuously scheduling the third preset number of times, generate a third warning message and send the third warning message to the relevant personnel.

[0100] The processing node monitoring module 92 is also used to log in to the ODS platform to schedule the corresponding unloading task when the unloading has not been completed; when the unloading is still not successful after continuously scheduling the fourth preset number of times, generate a fourth warning message and send the fourth warning message to the relevant personnel.

[0101] The processing node monitoring module 92 is also used to schedule the CHANNEL synchronization trigger to synchronize the data to the CHANNEL temporary table when it has not been synchronized to the CHANNEL temporary table; when it still has not been synchronized to the CHANNEL temporary table after continuously scheduling the fifth preset number of times, generate a fifth warning message and send the fifth warning message to the relevant personnel.

[0102] The processing node monitoring module 92 is also used to log in to the OGG platform to schedule the corresponding synchronization task when it has not been synchronized to the UM system; when it still has not been synchronized to the UM system after continuously scheduling the sixth preset number of times, generate a sixth warning message and send the sixth warning message to the relevant personnel.

[0103] It should be noted that for the content such as information interaction and execution process between the above units, since they are based on the same concept as the method embodiments of the present invention, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details will not be repeated here.

[0104] Figure 10 FIG. is a schematic structural diagram of a computer device provided in Embodiment X of the present invention. As Figure 10 shown, the computer device of this embodiment includes: at least one processor ( Figure 10 only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, it implements the steps in any of the above-mentioned method embodiments for item recommendation.

[0105] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 10 this is only an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include a network interface, a display screen, and an input device, etc.

[0106] The so-called processor may be a CPU, and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0107] The memory includes a readable storage medium, an internal memory, etc. Among them, the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be the hard disk of a computer device, and in some other embodiments, it can also be an external storage device of a computer device. For example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the computer device. Further, the memory can also include both the internal storage unit of the computer device and the external storage device. The memory is used to store the operating system, application programs, a BootLoader, data, and other programs, etc. The other programs such as the program code of a computer program, etc. The memory can also be used to temporarily store the data that has been output or will be output.

[0108] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of the functional unit of the target application program. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the foregoing method embodiment and will not be elaborated herein. If the integrated unit is implemented in the form of the functional unit of the target application program and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment method of the present invention, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and the distribution medium of the target application program. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0109] All or part of the processes in the above-mentioned embodiment method of the present invention can also be completed by a computer program product. When the computer program product runs on a computer device, the computer device can be made to execute the steps that can be implemented in the above-mentioned method embodiment.

[0110] In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of the target application program and electronic hardware. Whether these functions are executed in the form of hardware or the target application program depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0112] In the embodiments provided by the present invention, it should be understood that the disclosed device / computer device and method can be implemented in other ways. For example, the device / computer device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0114] The above has introduced in detail the data synchronization method, device, computer device and storage medium based on the monitoring of the processing node state provided by the present invention. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A data synchronization method based on monitoring the status of processing nodes, characterized in that, The method includes: Receiving a synchronization request from a data requester; In response to the data synchronization request, detecting whether the data to be synchronized has been synchronized to the public search space temporary table; If it has been synchronized to the public search space temporary table, detecting whether the data to be synchronized has completed data extraction; If the data extraction has been completed, detecting whether the Saturn file transfer task has been successfully scheduled; If the scheduling is successful, detecting whether the data has been unloaded; If the data has been unloaded, detecting whether the data has been synchronized to the CHANNEL temporary table; If it has been synchronized to the CHANNEL temporary table, detecting whether the data has been synchronized to the UM system; If it has been synchronized to the UM system, the data synchronization is completed.

2. The data synchronization method based on processing node status monitoring according to claim 1, wherein After the step of detecting whether the data to be synchronized has been synchronized to the public search space temporary table, the method further includes: If it has not been synchronized to the public search space temporary table, scheduling a CSS synchronization trigger to synchronize the data to the public search space temporary table; If it still has not been synchronized to the public search space temporary table after continuously scheduling a first preset number of times, generating a first warning message and sending the first warning message to the relevant personnel.

3. The data synchronization method based on processing node status monitoring according to claim 1, wherein, After the step of detecting whether the data to be synchronized has completed data extraction, the method further includes: If the data extraction has not been completed, logging in to the DS platform to schedule the corresponding data extraction task; If the data extraction still has not been completed after continuously scheduling a second preset number of times, generating a second warning message and sending the second warning message to the relevant personnel.

4. The data synchronization method based on processing node status monitoring according to claim 1, characterized in that After the step of detecting whether the Saturn file transfer task has been successfully scheduled, the method further includes: If the scheduling is not successful, logging in to the Saturn platform to schedule the corresponding file transfer task; If the Saturn file transfer task still has not been completed after continuously scheduling a third preset number of times, generating a third warning message and sending the third warning message to the relevant personnel.

5. The data synchronization method based on processing node status monitoring according to claim 1, characterized in that After the step of detecting whether the data has been unloaded, the method further includes: If the unloading has not been completed, logging in to the ODS platform to schedule the corresponding unloading task; If the unloading is still not successful after continuously scheduling a fourth preset number of times, generating a fourth warning message and sending the fourth warning message to the relevant personnel.

6. The data synchronization method based on processing node status monitoring according to claim 1, wherein After the step of detecting whether the data has been synchronized to the CHANNEL temporary table, the method further includes: If it has not been synchronized to the CHANNEL temporary table, scheduling a CHANNEL synchronization trigger to synchronize the data to the CHANNEL temporary table; If it still has not been synchronized to the CHANNEL temporary table after continuously scheduling a fifth preset number of times, generating a fifth warning message and sending the fifth warning message to the relevant personnel.

7. The data synchronization method based on the processing node status monitoring according to claim 1, characterized in that After the step of detecting whether the data has been synchronized to the UM system, the method further includes: If it has not been synchronized to the UM system, logging in to the OGG platform to schedule the corresponding synchronization task; If it still has not been synchronized to the UM system after continuously scheduling a sixth preset number of times, generating a sixth warning message and sending the sixth warning message to the relevant personnel.

8. A data synchronization device, characterized in that, The device includes: A receiving module for receiving a synchronization request from a data requester; The processing node monitoring module is used to respond to the data synchronization request, detect whether the data to be synchronized has been synchronized to the public search space temporary table. If it has been synchronized to the public search space temporary table, then detect whether the data extraction of the data to be synchronized has been completed. If the data extraction has been completed, then detect whether the Saturn file transfer task has been scheduled successfully. If the scheduling is successful, then detect whether the data has been unloaded. If the data has been unloaded, then detect whether the data has been synchronized to the CHANNEL temporary table. If it has been synchronized to the CHANNEL temporary table, then detect whether the data has been synchronized to the UM system.

9. An electronic device, characterized in that, The electronic device includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data synchronization method based on processing node status monitoring according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the data synchronization method based on processing node status monitoring according to any one of claims 1 to 7.

Images