Data updating method and device and related equipment

By generating and sending log information on the main computing node of the cloud relational database, and receiving the parsing status information from the computing node for verification, the data inconsistency caused by uncertain speed of parsing application logs from the computing node and the storage node is solved, and the data inconsistency is improved. Data update efficiency and consistency.

CN118939668APending Publication Date: 2024-11-12CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410994734.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In a cloud relational database, due to the uncertain speed of parsing application logs from the computing nodes and storage nodes, inconsistent data is read from the master computing nodes and from the computing nodes, thus reducing the data update efficiency.

Method used

Log information is generated at the main computing node and sent to N slave computing nodes, and the analysis status information of the slave computing node is received. By verifying this information, it determines whether the slave computing node has successfully updated the target data, and resends the log information according to the verification result. Ensure consistency of all data from the compute nodes.

Benefits of technology

Through this method, the problem of low data update efficiency is solved, ensuring synchronous update of target data from the computing node, and improving data consistency and update efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939668A_ABST
    Figure CN118939668A_ABST
Patent Text Reader

Abstract

The invention provides a data updating method and device and related equipment, which are applied to a main computing node, and the method comprises the following steps: generating log information under the condition that target data included in the main computing node is updated; the log information is sent to N slave computing nodes, so that the slave computing nodes update stored target data based on the received log information; receiving N pieces of analysis state information sent by the N slave computing nodes; and verifying the N pieces of analysis state information to generate N verification results. According to the data updating method and device, after the updated log information is sent to the N computing nodes, whether the N computing nodes update the target data or not is verified by receiving the analysis state information corresponding to the N computing nodes, and the problem that in the prior art, the data updating efficiency is low is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of cloud computing, and in particular to a data updating method, apparatus, and related equipment. Background Art

[0002] Compared with traditional databases, cloud relational databases use a computing and storage separation architecture to maintain high availability of the database. That is, a one-master-multiple-slave distributed architecture based on shared storage is usually used to provide read and write services. Generally, the master computing node is used as a read-write computing node (RW) to process read and write requests, and the slave computing node is used as a read-only computing node (RO) to process read-only requests. All storage tasks are pushed down to the storage node (SE). However, although master-slave data synchronization can be achieved through redo logs, the speed of parsing application logs from the computing node and storage node is uncertain, especially during the execution of the data definition language by the master computing node, if the synchronization of file reading from the computing node and storage node is not controlled well, inconsistent data will be read from the computing node and storage node, resulting in low data update efficiency of the master computing node and the slave computing node. Summary of the invention

[0003] The embodiments of the present application provide a data updating method, apparatus and related equipment to solve the problem of low data updating efficiency in the prior art.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a data update method, which is applied to a master computing node, and the method includes:

[0006] When the target data included in the main computing node is updated, generating log information, the log information including the update data of the target data;

[0007] Sending the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data;

[0008] Receiving N parsing status information sent by the N slave computing nodes, the N parsing status information corresponding to the N slave computing nodes one by one, the parsing status information being used to indicate whether the corresponding slave computing node successfully parses the log information;

[0009] The N parsing status information are verified to generate N verification results, wherein the N verification results correspond one-to-one to the N slave computing nodes, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data.

[0010] Optionally, the verifying the N pieces of parsing status information to generate N verification results includes:

[0011] Constructing a log verification object, the log verification object including at least one of the following: update operation information, count information, and N execution information, wherein the update operation information is used to characterize the operation type of the update operation corresponding to the log information, the count information is used to indicate the number of slave computing nodes that have fed back the parsing status information, and the execution information is used to characterize whether the slave computing node has successfully updated the stored target data;

[0012] The N pieces of parsing status information are verified based on the log verification object to generate the N verification results.

[0013] Optionally, the receiving the N pieces of parsing status information sent by the N slave computing nodes includes:

[0014] In the process of receiving the N parsing status information, generating the counting information;

[0015] The build log verification object includes:

[0016] Determine update operation information according to the log information;

[0017] When the value of the count information is N, determine N execution information based on the N parsing state information, the N parsing state information corresponds one-to-one to the N execution information, and when the master computing node receives the N parsing state information, the value of the count information is N;

[0018] The log verification object is generated according to the update operation information and N execution information.

[0019] Optionally, after verifying the N parsing status information based on the log verification object and generating the N verification results, the method further includes:

[0020] If the target verification result indicates that the slave computing node corresponding to the target data has not updated the target data, an error log is generated, and the target verification result is any one of the N verification results;

[0021] Determine a target slave computing node based on the error log, the target slave computing node corresponding to the target verification result;

[0022] Resend the log information to the target slave computing node.

[0023] Optionally, sending the log information to N slave computing nodes includes:

[0024] Sending the log information to a storage node;

[0025] After receiving the storage completion information sent by the storage node, the log information is sent to N slave computing nodes based on the storage completion information.

[0026] Optionally, after receiving the storage completion information sent by the storage node, after sending the log information to N slave computing nodes based on the storage completion information, the method further includes:

[0027] Generate obstruction information based on the storage completion information, where the obstruction information is used to prevent the main computing node from generating other log information except the log information;

[0028] In the process of receiving the N pieces of parsing status information, after generating the counting information, the method further includes:

[0029] When the value of the count information is N, the blocking information is deleted.

[0030] In a second aspect, an embodiment of the present application further provides a data updating device, which is applied to a main computing node, and the device includes:

[0031] A generating module, configured to generate log information when there is an update of the target data included in the main computing node, wherein the log information includes the update data of the target data;

[0032] A sending module, used for sending the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data;

[0033] A receiving module, used to receive N parsing status information sent by the N slave computing nodes, wherein the N parsing status information corresponds to the N slave computing nodes one by one, and the parsing status information is used to indicate whether the corresponding slave computing node successfully parses the log information;

[0034] A verification module is used to verify the N parsing status information and generate N verification results, wherein the N verification results correspond one-to-one to the N slave computing nodes, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data.

[0035] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps of the method described in the first aspect above.

[0036] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the method described in the first aspect above.

[0037] In a fifth aspect, an embodiment of the present application further provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps in the method described in the first aspect.

[0038] The present application provides a data updating method, apparatus and related equipment, which are applied to a master computing node, the method comprising: generating log information when target data included in the master computing node is updated, the log information comprising updated data of the target data; sending the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data; receiving N parsing status information sent by the N slave computing nodes, the N parsing status information corresponding one-to-one to the N slave computing nodes, the parsing status information being used to characterize whether the corresponding slave computing node has successfully parsed the log information; verifying the N parsing status information, generating N verification results, the N verification results corresponding one-to-one to the N slave computing nodes, the verification results being used to indicate whether the corresponding slave computing node has updated the target data. The present application solves the problem of low data update efficiency in the prior art by sending updated log information to N computing nodes and then receiving parsing status information corresponding to the N computing nodes to verify whether the N computing nodes have updated the target data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 It is a flowchart of a data updating method provided in an embodiment of the present application;

[0041] Figure 2 It is an interactive schematic diagram provided in an embodiment of the present application;

[0042] Figure 3 is a structural diagram of a data updating device provided in an embodiment of the present application;

[0043] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing the inclusion of separate A, separate B, separate C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations that A, B and C all exist.

[0046] See also Figure 1 , Figure 1 is a flow chart of a data updating method provided in an embodiment of the present application, such as Figure 1 As shown, the data update method is applied to the main computing node and may include the following steps:

[0047] Step 101: When target data included in the main computing node is updated, log information is generated, where the log information includes update data of the target data.

[0048] In this embodiment, the master computing node is generally a read-write computing node (RW) that processes read-write requests, while the slave computing node is a read-only computing node (RO) that processes read-only requests. Multiple slave computing nodes are respectively connected to the master computing node for communication. Generally, storage tasks are stored in the storage node (SE). When the master computing node needs to update data, that is, when the target data is updated, log information needs to be generated. The log information is the update data for updating the target data.

[0049] In addition, it should be noted that the data update is generally a data definition language (DDL) data update, and corresponding log information is generated according to the execution situation. Specifically, the data definition language (DDL) is a database language used to define the table structure, which can be used to create, modify, and delete the structure of the data table, thereby realizing the control and management of the table. Generally, DDL operations involve the creation, renaming, deletion, and replacement of metadata files, as well as the creation, renaming, and deletion of data files. The operations of data files will write redo logs, and the operations related to metadata files need to add log types at the server layer to achieve the synchronization of metadata files. The one-master-multiple-slave distributed architecture based on shared storage involves operations such as reading and writing metadata files and data files in both slave computing nodes and storage nodes. Since the files are pushed down to the storage node for storage, and the master and slave computing nodes share a file, during the execution of DDL by the master computing node, the slave computing node and the storage node perform log parsing and application at their own pace. The slave computing node updates the file memory information according to the file log type, and the storage node not only updates the file memory information but also performs the real physical operation of the file.

[0050] Step 102: Send the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data.

[0051] In this embodiment, after the log information is generated, the log information needs to be sent to N slave computing nodes respectively, so that the N slave computing nodes can update the stored target data accordingly according to the log information. Specifically, each slave computing node will periodically request log information from the master computing node, and when it is obtained that log information is generated, it will send a request for obtaining the log information to the master computing node, so that the master computing node sends the log information to the N slave computing nodes.

[0052] Step 103: Receive N parsing status information sent by the N slave computing nodes, wherein the N parsing status information corresponds one-to-one to the N slave computing nodes, and the parsing status information is used to indicate whether the corresponding slave computing node successfully parses the log information.

[0053] In this embodiment, after receiving the log information, N slave computing nodes parse the log information respectively and generate parsing status information according to different parsing situations, wherein the parsing status information is used to indicate whether the corresponding slave computing node has successfully parsed the log information, for example, the parsing is successful or failed.

[0054] It should be noted that, in the present embodiment, the N computing nodes will return to the main computing node one by one, and the main computing node will perform real-time statistics on the received parsing status information, so that the number of slave computing nodes that have currently parsed the log information can be accurately known. Exemplarily, the parsing status information can be counted, and the count is increased by one after receiving one parsing status information, until the counting stops after all N parsing status information are received. In addition, there may be a situation where a slave computing node fails, so that it is unable to receive all N parsing status information. At this time, a certain time threshold can be set. If the time threshold is exceeded, if not all N parsing status information are received, it can be assumed that other slave computing nodes that have not returned parsing status information are in a faulty state.

[0055] Step 104: verify the N parsing status information to generate N verification results, wherein the N verification results correspond one-to-one to the N slave computing nodes, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data.

[0056] In this embodiment, after receiving all N parsing status information, it is necessary to verify the N parsing status information one by one to determine whether the corresponding slave computing node has updated the target data. In addition, through the N verification results, the synchronization status between the N slave computing nodes and the main computing node can be determined. It should be noted that a successful verification means that the content included in the reply message of the slave computing node is completely matched with the information of the DDL operation executed by the main computing node.

[0057] The present application provides a data update method, the method comprising: generating log information when the target data included in the master computing node is updated, the log information including the updated data of the target data; sending the log information to N slave computing nodes so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data; receiving N parsing status information sent by the N slave computing nodes, the N parsing status information corresponding to the N slave computing nodes one-to-one, and the parsing status information is used to characterize whether the corresponding slave computing node successfully parses the log information; verifying the N parsing status information, generating N verification results, the N verification results corresponding to the N slave computing nodes one-to-one, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data. The present application verifies whether the target data is updated by the N computing nodes after sending the updated log information to the N computing nodes, and receives the parsing status information corresponding to the N computing nodes, thereby solving the problem of low data update efficiency in the prior art.

[0058] In some feasible implementations, optionally, verifying the N parsing status information to generate N verification results includes:

[0059] Constructing a log verification object, the log verification object including at least one of the following: update operation information, count information, and N execution information, wherein the update operation information is used to characterize the operation type of the update operation corresponding to the log information, the count information is used to indicate the number of slave computing nodes that have fed back the parsing status information, and the execution information is used to characterize whether the slave computing node has successfully updated the stored target data;

[0060] The N pieces of parsing status information are verified based on the log verification object to generate the N verification results.

[0061] In this embodiment, if Figure 2 As shown, Figure 2 This is an interaction diagram in the present application, in which the master computing node is respectively connected to the slave computing node and the storage node for communication. The master computing node generates corresponding file operation-related logs during the execution of DDL, builds a log verification object for this file operation based on the file operation information, and adds the log verification object to the global DDL management module.

[0062] Among them, the log verification object is composed of at least one of the following, including: update operation information, counting information and N execution information. It should be noted that the update operation information is the type of operation, such as the creation, renaming, deletion, replacement of metadata files and the creation, renaming and deletion of data files. The file operation information includes: log type, tablespace number, request number, DDL type, file name. The counting information is a statistic performed after receiving the parsing status information fed back from the computing node, wherein the number of slave computing nodes that have fed back the parsing status information can be accurately determined through the counting information. The execution information is used to characterize whether each slave computing node has successfully updated the stored target data. Therefore, the N parsing status information is verified through the log verification object, thereby generating corresponding N verification results.

[0063] Optionally, the receiving the N pieces of parsing status information sent by the N slave computing nodes includes:

[0064] In the process of receiving the N parsing status information, generating the counting information;

[0065] The build log verification object includes:

[0066] Determine update operation information according to the log information;

[0067] When the value of the count information is N, determine N execution information based on the N parsing state information, the N parsing state information corresponds one-to-one to the N execution information, and when the master computing node receives the N parsing state information, the value of the count information is N;

[0068] The log verification object is generated according to the update operation information and N execution information.

[0069] In this embodiment, the main computing node receives N parsing status information in sequence. If none of them is received, the value of the counting information is 0. If all of them are received, the value of the counting information is N. After receiving one parsing status information, the counting information is increased by one until the value of the counting information is N.

[0070] It should also be noted that the update operation information is determined by the log information, and the N execution information is determined by the N parsing status information. The log verification object, the global DDL management module saves the logs of file-related operations during all DDL requests of the main computing node. Each file-related operation log corresponds to a unique log verification object. The file-related operation logs generated by multiple concurrent DDL operations do not affect each other. The global DDL management module stores and manages all log verification objects in a hash table data structure. The key of the hash table is composed of the file operation information to which it belongs, and the value is the log verification object. The log verification object and the global DDL management module are both stored in the main computing node.

[0071] Optionally, after verifying the N parsing status information based on the log verification object and generating the N verification results, the method further includes:

[0072] If the target verification result indicates that the slave computing node corresponding to the target data has not updated the target data, an error log is generated, and the target verification result is any one of the N verification results;

[0073] Determine a target slave computing node based on the error log, the target slave computing node corresponding to the target verification result;

[0074] Resend the log information to the target slave computing node.

[0075] In this embodiment, some slave computing nodes may not be able to accurately update the target data, and therefore, an error log is generated to indicate that a slave computing node has failed to update. Whether to generate an error log is determined based on a target verification result, wherein the target verification result is any one of the N verification results.

[0076] Thus, the slave computing nodes that have not been updated normally are determined through the error log, and the log information is resent to the target slave computing nodes, thereby achieving that all slave computing nodes can successfully update the log information.

[0077] Optionally, sending the log information to N slave computing nodes includes:

[0078] Sending the log information to a storage node;

[0079] After receiving the storage completion information sent by the storage node, the log information is sent to N slave computing nodes based on the storage completion information.

[0080] In this embodiment, the log information needs to be sent to the storage node for log persistence. The log persistence is achieved through the real-time communication interface synchronization between the main computing node and the storage node. The main computing node log is successfully stored on the storage node for persistence. The redo log buffer stores all logs generated before the main computing node is stored on the disk. The logs are stored in 512-byte blocks and are only provided in real time to the slave computing node.

[0081] Optionally, after receiving the storage completion information sent by the storage node, after sending the log information to N slave computing nodes based on the storage completion information, the method further includes:

[0082] Generate obstruction information based on the storage completion information, where the obstruction information is used to prevent the main computing node from generating other log information except the log information;

[0083] In the process of receiving the N pieces of parsing status information, after generating the counting information, the method further includes:

[0084] When the value of the count information is N, the blocking information is deleted.

[0085] In this embodiment, the slave computing node periodically sends a log request to the master computing node to obtain the latest redo log. When the slave computing node receives this log and successfully parses this log, it will feedback the DDL reply information to the master computing node. After the master computing node receives the DDL reply information from the slave computing node, it builds a log verification object according to the DDL reply information, and then goes to the global DDL management module for verification. If the verification is unsuccessful, an error log is reported. If the verification is successful, the next step is performed according to the DDL execution result. If the DDL execution result is unsuccessful, the master computing node will report an error log. If the DDL execution result is successful, the reply count of this log verification object is increased. When the reply count of this log verification object is greater than or equal to the number of currently surviving slave computing nodes, a signal is sent to the thread executing DDL to unblock it and continue to execute the logic behind DDL until DDL ends.

[0086] The DDL reply information of the slave computing node includes: slave computing node number, log type, tablespace number, request number, DDL type, file name, and DDL execution result. The master computing node constructs a log verification object based on the DDL reply information, increases the reply count after successful verification, and decides whether to send a signal to release the DDL thread blockage based on whether the DDL execution result in the DDL reply information and the reply count is greater than or equal to the current number of surviving slave computing nodes are met at the same time; only when both conditions are met can the DDL thread blockage be released; otherwise, the DDL thread blockage is not released. Wherein, successful verification means that the content included in the DDL reply information completely matches the information of the master node executing the DDL operation.

[0087] During the execution of DDL by the main computing node of this application, the corresponding file operation-related logs are generated, the log verification object of this file operation is constructed according to the information of this file operation, and the object is added to the global DDL management module. After that, the main computing node persists the file-related logs to the storage node, and saves the logs to the local redo log buffer for a period of time to meet the needs of obtaining logs from the computing node. After that, the main computing node blocks and waits for the information reply parsed from the computing node to this log.

[0088] The slave compute node periodically sends log requests to the master compute node to obtain the latest redo log. When the slave compute node receives this log and successfully parses it, it will feedback DDL reply information to the master compute node. After the master compute node receives the DDL reply information from the slave compute node, it builds a log verification object based on the DDL reply information, and then goes to the global DDL management module for verification. If the verification fails, an error log is reported. If the verification succeeds, the next step is performed according to the DDL execution result. If the DDL execution result is unsuccessful, the master compute node will report an error log. If the DDL execution result is successful, the reply count of this log verification object is increased. When the reply count of this log verification object is greater than or equal to the number of currently surviving slave compute nodes, a signal is sent to the thread executing DDL to unblock it and continue to execute the logic behind DDL until DDL ends.

[0089] In addition, this application is applied to the research of HTAP technology of cloud native database, which can prevent the computing node from accessing a non-existent file too quickly or too slowly; DDL reply information can uniquely mark a file operation log to meet the needs of concurrent scenario execution; it can still be applicable in high-availability scenarios such as after the computing node hangs up; the log is stored in a hash table data structure for easy and quick query.

[0090] The present application solves the problem of low data update efficiency in the prior art by sending updated log information to N computing nodes and then receiving parsing status information corresponding to the N computing nodes to verify whether the N computing nodes have updated the target data.

[0091] See also Figure 3 , Figure 3 is a structural diagram of a data updating device provided in an embodiment of the present application. Figure 3 As shown, the data updating device 300 is applied to the main computing node, including:

[0092] A generating module 310, configured to generate log information when there is an update of the target data included in the master computing node, wherein the log information includes update data of the target data;

[0093] A sending module 320, configured to send the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data;

[0094] A receiving module 330 is used to receive N parsing status information sent by the N slave computing nodes, wherein the N parsing status information corresponds to the N slave computing nodes one by one, and the parsing status information is used to indicate whether the corresponding slave computing node successfully parses the log information;

[0095] The verification module 340 is used to verify the N parsing status information and generate N verification results, wherein the N verification results correspond one-to-one to the N slave computing nodes, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data.

[0096] Optionally, the verification module 340 includes:

[0097] A construction submodule is used to construct a log verification object, wherein the log verification object includes at least one of the following: update operation information, count information, and N execution information, wherein the update operation information is used to characterize the operation type of the update operation corresponding to the log information, the count information is used to indicate the number of slave computing nodes that have fed back the parsing status information, and the execution information is used to characterize whether the slave computing node has successfully updated the stored target data;

[0098] The verification submodule is used to verify the N parsing status information based on the log verification object and generate the N verification results.

[0099] Optionally, the receiving module 330 includes:

[0100] A generating submodule, configured to generate the counting information in the process of receiving the N parsing status information;

[0101] Build submodules, including:

[0102] A first determining unit, configured to determine update operation information according to the log information;

[0103] a second determining unit, configured to determine, when the value of the count information is N, N execution information based on the N parsing state information, the N parsing state information corresponding one-to-one to the N execution information, and when the master computing node receives the N parsing state information, the value of the count information is N;

[0104] A generating unit is used to generate the log verification object according to the update operation information and N execution information.

[0105] Optionally, also include:

[0106] an error generation module, configured to generate an error log when a target verification result indicates that a slave computing node corresponding to the target data has not updated the target data, wherein the target verification result is any one of the N verification results;

[0107] A determination module, configured to determine a target slave computing node based on the error log, wherein the target slave computing node corresponds to the target verification result;

[0108] The resending module is used to resend the log information to the target slave computing node.

[0109] Optionally, the sending module 320 includes:

[0110] A first sending submodule, used for sending the log information to a storage node;

[0111] The second sending submodule is used to send the log information to N slave computing nodes based on the storage completion information after receiving the storage completion information sent by the storage node.

[0112] Optionally, also include:

[0113] An obstruction generation module, configured to generate obstruction information based on the storage completion information, wherein the obstruction information is used to prevent the main computing node from generating other log information except the log information;

[0114] The blocking deletion module is used to delete the blocking information when the value of the counting information is N.

[0115] The present application solves the problem of low data update efficiency in the prior art by sending updated log information to N computing nodes and then receiving parsing status information corresponding to the N computing nodes to verify whether the N computing nodes have updated the target data.

[0116] The present application also provides an electronic device. Figure 4 , the electronic device may include a processor 401, a memory 402, and a program 4021 stored in the memory 402 and executable on the processor 401.

[0117] When the program 4021 is executed by the processor 401, it can achieve Figure 1 Any step in the corresponding method embodiment:

[0118] When the target data included in the main computing node is updated, generating log information, the log information including the update data of the target data;

[0119] Sending the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data;

[0120] Receiving N parsing status information sent by the N slave computing nodes, the N parsing status information corresponding to the N slave computing nodes one by one, the parsing status information being used to indicate whether the corresponding slave computing node successfully parses the log information;

[0121] The N parsing status information are verified to generate N verification results, wherein the N verification results correspond one-to-one to the N slave computing nodes, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data.

[0122] Optionally, the verifying the N pieces of parsing status information to generate N verification results includes:

[0123] Constructing a log verification object, the log verification object including at least one of the following: update operation information, count information, and N execution information, wherein the update operation information is used to characterize the operation type of the update operation corresponding to the log information, the count information is used to indicate the number of slave computing nodes that have fed back the parsing status information, and the execution information is used to characterize whether the slave computing node has successfully updated the stored target data;

[0124] The N pieces of parsing status information are verified based on the log verification object to generate the N verification results.

[0125] Optionally, the receiving the N pieces of parsing status information sent by the N slave computing nodes includes:

[0126] In the process of receiving the N parsing status information, generating the counting information;

[0127] The build log verification object includes:

[0128] Determine update operation information according to the log information;

[0129] When the value of the count information is N, determine N execution information based on the N parsing state information, the N parsing state information corresponds one-to-one to the N execution information, and when the master computing node receives the N parsing state information, the value of the count information is N;

[0130] The log verification object is generated according to the update operation information and N execution information.

[0131] Optionally, after verifying the N parsing status information based on the log verification object and generating the N verification results, the method further includes:

[0132] If the target verification result indicates that the slave computing node corresponding to the target data has not updated the target data, an error log is generated, and the target verification result is any one of the N verification results;

[0133] Determine a target slave computing node based on the error log, the target slave computing node corresponding to the target verification result;

[0134] Resend the log information to the target slave computing node.

[0135] Optionally, sending the log information to N slave computing nodes includes:

[0136] Sending the log information to a storage node;

[0137] After receiving the storage completion information sent by the storage node, the log information is sent to N slave computing nodes based on the storage completion information.

[0138] Optionally, after receiving the storage completion information sent by the storage node, after sending the log information to N slave computing nodes based on the storage completion information, the method further includes:

[0139] Generate obstruction information based on the storage completion information, where the obstruction information is used to prevent the main computing node from generating other log information except the log information;

[0140] In the process of receiving the N pieces of parsing status information, after generating the counting information, the method further includes:

[0141] When the value of the count information is N, the blocking information is deleted.

[0142] The present application solves the problem of low data update efficiency in the prior art by sending updated log information to N computing nodes and then receiving parsing status information corresponding to the N computing nodes to verify whether the N computing nodes have updated the target data.

[0143] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned data updating method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0144] The embodiment of the present application further provides a computer program product, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the various processes of the above-mentioned data update method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0145] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0147] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A data updating method, characterized in that: Applied to the main computing node, the method includes: When the target data included in the main computing node is updated, generating log information, the log information including the update data of the target data; Sending the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data; Receiving N parsing status information sent by the N slave computing nodes, the N parsing status information corresponding to the N slave computing nodes one by one, the parsing status information being used to indicate whether the corresponding slave computing node successfully parses the log information; The N parsing status information are verified to generate N verification results, wherein the N verification results correspond one-to-one to the N slave computing nodes, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data.

2. The method according to claim 1, characterized in that The verifying the N parsing status information to generate N verification results includes: Constructing a log verification object, the log verification object including at least one of the following: update operation information, count information, and N execution information, wherein the update operation information is used to characterize the operation type of the update operation corresponding to the log information, the count information is used to indicate the number of slave computing nodes that have fed back the parsing status information, and the execution information is used to characterize whether the slave computing node has successfully updated the stored target data; The N pieces of parsing status information are verified based on the log verification object to generate the N verification results.

3. The method according to claim 2, characterized in that The receiving the N parsing status information sent by the N slave computing nodes includes: In the process of receiving the N parsing status information, generating the counting information; The build log verification object includes: Determine update operation information according to the log information; When the value of the count information is N, determine N execution information based on the N parsing state information, the N parsing state information corresponds one-to-one to the N execution information, and when the master computing node receives the N parsing state information, the value of the count information is N; The log verification object is generated according to the update operation information and N execution information.

4. The method according to claim 2, characterized in that: After verifying the N parsing status information based on the log verification object and generating the N verification results, the method further includes: If the target verification result indicates that the slave computing node corresponding to the target data has not updated the target data, an error log is generated, and the target verification result is any one of the N verification results; Determine a target slave computing node based on the error log, the target slave computing node corresponding to the target verification result; Resend the log information to the target slave computing node.

5. The method according to claim 3, characterized in that: The sending the log information to N slave computing nodes includes: Sending the log information to a storage node; After receiving the storage completion information sent by the storage node, the log information is sent to N slave computing nodes based on the storage completion information.

6. The method according to claim 5, characterized in that After receiving the storage completion information sent by the storage node, the method further includes sending the log information to N slave computing nodes based on the storage completion information: Generate obstruction information based on the storage completion information, where the obstruction information is used to prevent the main computing node from generating other log information except the log information; In the process of receiving the N pieces of parsing status information, after generating the counting information, the method further includes: When the value of the count information is N, the blocking information is deleted.

7. A data updating device, characterized in that: Applied to a main computing node, the device comprises: A generating module, configured to generate log information when there is an update of the target data included in the main computing node, wherein the log information includes the update data of the target data; A sending module, used for sending the log information to N slave computing nodes, so that the slave computing nodes update the stored target data based on the received log information, wherein N is a positive integer, and each of the N slave computing nodes stores the target data; A receiving module, used to receive N parsing status information sent by the N slave computing nodes, wherein the N parsing status information corresponds to the N slave computing nodes one by one, and the parsing status information is used to indicate whether the corresponding slave computing node successfully parses the log information; A verification module is used to verify the N parsing status information and generate N verification results, wherein the N verification results correspond one-to-one to the N slave computing nodes, and the verification results are used to indicate whether the corresponding slave computing node has updated the target data.

8. An electronic device comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the data updating method as described in any one of claims 1 to 6.

9. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the data updating method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the data updating method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Main and slave node data synchronization method in Redis

    CN106657169A

  • Method and device for synchronizing data consistency

    CN107426265A

  • Database synchronization method, device, system and equipment and medium

    CN114077592A

  • Memory key value erasure code-oriented hybrid data update method, and storage medium

    WO2023082556A1