Data processing method and device, related equipment and storage medium

Abstract

The application discloses a data processing method and device, a sending end device, a receiving end device and a storage medium. The method comprises the following steps: the sending end device acquires at least one instruction; the acquired at least one instruction is put into a first queue; an instruction is read from the first queue according to the order of entering the queue, corresponding data of the instruction is acquired in response to the read instruction, and the instruction and the corresponding data are put into a second queue; the instruction and the corresponding data in the second queue are synchronized to the receiving end device according to the order of entering the queue; the receiving end device receives the sequentially sent at least one instruction and the corresponding data; for each instruction in the at least one instruction, the legality of the instruction is determined, in the case that the instruction is legal, the corresponding data is stored in a tree structure mode in response to the instruction; and in the case that the instruction is not legal, the instruction and the corresponding data are cached. The data synchronization instruction is sent through the first queue and the second queue, and the accuracy of the tree structure data on the receiving end side is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a data processing method and device, a sending end device, a receiving end device and a storage medium. BACKGROUND

[0002] At present, tree structure data is widely used in different business scenarios, such as administrative division management, enterprise organization management or network device management scenarios. In application systems with the same business type, tree structure data needs to be synchronized to realize the sharing of key data.

[0003] In the related art, the data sending party sends the data that needs to be synchronized in the tree structure data to the data receiving party through network communication transmission, and the data receiving party synchronizes the corresponding tree structure data according to the received data. In the process of data synchronization, when the data frequently changes, the data sending party has a large processing pressure and it is difficult to guarantee the timing of data synchronization, which may cause the order of the data received by the data receiving party to be inconsistent with that of the data sending party, thereby affecting the accuracy of the tree structure data of the data receiving party, and the accuracy of the tree structure data of the receiving party is low. SUMMARY

[0004] To solve the problems in the related art, the embodiments of the present application provide a data processing method and device, a sending end device, a receiving end device and a storage medium.

[0005] The technical scheme of the embodiments of the present application is implemented as follows:

[0006] The embodiments of the present application provide a data processing method applied to a sending end device, comprising:

[0007] Obtaining at least one instruction, the instruction being used to indicate the sending of data for synchronization to a receiving end device; the at least one instruction comprising a first type of instruction and / or a second type of instruction, the first type of instruction being used to subscribe to the data of a specific node in a tree structure, and the second type of instruction indicating the data whose increment changes;

[0008] Putting the obtained at least one instruction into a first queue;

[0009] Reading one instruction from the first queue in the order of entry, obtaining the data corresponding to the corresponding instruction in response to the read instruction, and putting the corresponding instruction and the corresponding data into a second queue; wherein the data corresponding to one instruction is put into the second queue, and then the next instruction is read from the first queue;

[0010] Synchronizing the corresponding instruction and the corresponding data in the second queue to the receiving end device in the order of entry.

[0011] In the above solution, the type of the read instruction is determined from the first queue;

[0012] When the type of the read instruction contains the first type, the full data under the subscription node is obtained by using the tree structure data, and the first information, the second information and the full data under the subscription node corresponding to the read instruction are put into the second queue, the first information indicating the start of the full data under the subscription node, and the second information indicating the end of the full data under the subscription node;

[0013] When the type of the read instruction contains the second type, the data corresponding to the read instruction is obtained from the first queue.

[0014] In the above solution, the method further comprises:

[0015] The subscription reload instruction sent by the receiving end device is obtained, and the subscription reload instruction is used to resubscribe the data of a specific node in the tree structure;

[0016] The subscription reload instruction is put into the first queue.

[0017] Embodiments of the present application also provide a data processing method, applied to a receiving end device, comprising:

[0018] At least one instruction and corresponding data sent by a sending end device in sequence are received; the instruction is used to indicate that data for synchronization is sent to the receiving end device; the at least one instruction contains a first type of instruction and / or a second type of instruction, the first type of instruction is used to subscribe the data of a specific node in a tree structure, and the second type of instruction indicates that the data of which the synchronization increment changes;

[0019] For each instruction in the at least one instruction, the legality of the corresponding instruction is determined, in the case that the corresponding instruction is legal, the corresponding data is stored in a tree structure mode in response to the corresponding instruction; and in the case that the corresponding instruction is not legal, the corresponding instruction and the corresponding data are cached.

[0020] In the above solution, the legality of the corresponding instruction is determined, comprising:

[0021] The node corresponding to the corresponding instruction is verified;

[0022] When the verification is successful, it is determined that the corresponding instruction is legal;

[0023] When the verification fails, it is determined that the corresponding instruction is not legal.

[0024] In the above solution, the method further comprises:

[0025] determining whether the received data contains first information, and determining that the type of the corresponding instruction contains a first type when it is determined that the received data contains the first information; the first information indicating the start of full data under a subscription node.

[0026] In the foregoing solution, the method further includes:

[0027] determining whether the received data contains first information, and determining that the type of the corresponding instruction contains a second type when it is determined that the received data does not contain the first information; the first information indicating the start of full data under a subscription node.

[0028] In the foregoing solution, the type of the corresponding instruction contains a first type, and the method further includes:

[0029] in a case where the corresponding instruction is illegal, determining and caching all node-related information corresponding to a check failure of the corresponding instruction;

[0030] initiating a subscription reloading instruction to the sending end device based on the cached all node-related information; the subscription reloading instruction being used to resubscribe data of a specific node in a tree structure.

[0031] In the foregoing solution,

[0032] receiving second information; the second information indicating the end of full data under a subscription node;

[0033] initiating a subscription reloading instruction to the sending end device based on the cached all node-related information.

[0034] In the foregoing solution, the type of the corresponding instruction contains a second type, and the method further includes:

[0035] in a case where the corresponding instruction is illegal, determining first node-related information corresponding to a check failure of the corresponding instruction;

[0036] initiating the subscription reloading instruction to the sending end device based on the first node-related information; the subscription reloading instruction being used to resubscribe data of a specific node in a tree structure.

[0037] Embodiments of the present application further provide a data processing apparatus, which is arranged on a sending end device and includes:

[0038] an obtaining unit, configured to obtain at least one instruction, the instruction being used to indicate sending data for synchronization to a receiving end device; the at least one instruction containing a first type of instruction and / or a second type of instruction, the first type of instruction being used to subscribe data of a specific node in a tree structure, and the second type of instruction indicating data of which a synchronization increment has changed;

[0039] a first processing unit, configured to put the obtained at least one instruction into a first queue;

[0040] a second processing unit, configured to read one instruction from the first queue in the order of entering the first queue, obtain data corresponding to the read instruction, and put the corresponding instruction and the corresponding data into a second queue; wherein the data corresponding to one instruction is put into the second queue before the next instruction is read from the first queue;

[0041] a synchronization unit, configured to synchronize the corresponding instruction and the corresponding data in the second queue to a receiving end device in the order of entering the first queue.

[0042] Embodiments of the present application also provide a data processing apparatus arranged on a receiving end device, comprising:

[0043] a receiving unit, configured to receive at least one instruction and corresponding data sent by a sending end device in sequence; the instruction is used to instruct the sending of data for synchronization to the receiving end device; the at least one instruction comprises a first type of instruction and / or a second type of instruction; the first type of instruction is used to subscribe to data of a specific node in a tree structure; the second type of instruction instructs data whose synchronization increment changes;

[0044] a third processing unit, configured to determine the legality of each instruction in the at least one instruction, and in the case that the corresponding instruction is legal, store the corresponding data in a tree structure manner in response to the corresponding instruction;

[0045] a fourth processing unit, configured to cache the corresponding instruction and the corresponding data in the case that the corresponding instruction is not legal.

[0046] Embodiments of the present application also provide a sending end device, comprising a first processor and a first memory for storing a computer program capable of running on the processor,

[0047] wherein the first processor is configured to execute the steps of any of the methods on the sending end device side when running the computer program.

[0048] Embodiments of the present application also provide a receiving end device, comprising a second processor and a second memory for storing a computer program capable of running on the processor,

[0049] wherein the second processor is configured to execute the steps of any of the methods on the receiving end device side when running the computer program.

[0050] The embodiment of the present application further provides a storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of any one of the methods of the sending end device or to implement the steps of any one of the methods of the receiving end device.

[0051] The data processing method, device, sending end device, receiving end device and storage medium provided by the embodiment of the present application are as follows: the sending end device acquires at least one instruction, the instruction being used to indicate sending data for synchronization to the receiving end device; the acquired at least one instruction is put into a first queue; one instruction is read from the first queue in the order of entering the queue, corresponding data of the read instruction is acquired, and the corresponding instruction and the corresponding data are put into a second queue; the corresponding instruction and the corresponding data in the second queue are synchronized to the receiving end device in the order of entering the queue; the receiving end device receives at least one instruction and corresponding data sent by the sending end device in sequence; for each instruction in the at least one instruction, the legality of the corresponding instruction is determined, in the case that the corresponding instruction is legal, the corresponding data is stored in a tree structure mode in response to the corresponding instruction; and in the case that the corresponding instruction is not legal, the corresponding instruction and the corresponding data are buffered. The scheme provided by the embodiment of the present application is as follows: after the sending end device acquires the instruction for data synchronization, the instruction is processed in the order of entering the queue through the first queue and the second queue, and is sent to the receiving end device, that is, the instruction is processed in order through two levels of queues, so that the receiving end device can receive the synchronized data in sequence and store the data in a tree structure mode. In this way, the timing of the tree structure data synchronization is guaranteed, and the accuracy of the tree structure data on the receiving end side is improved. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The first data processing method flowchart of the embodiment of the present application is as follows:

[0053] Figure 2 The second data processing method flowchart of the embodiment of the present application is as follows:

[0054] Figure 3 The instruction verification method structure diagram of the first type of the embodiment of the present application is as follows:

[0055] Figure 4 The third data processing method flowchart of the embodiment of the present application is as follows:

[0056] Figure 5 The tree structure data synchronization system architecture diagram of the application embodiment is as follows:

[0057] Figure 6 The tree structure data synchronization method flowchart of the application embodiment is as follows

[0058] Figure 7 Figure 1 is a schematic diagram of a first data processing device structure according to an embodiment of the present application;

[0059] Figure 8 Figure 2 is a schematic diagram of a second data processing device structure according to an embodiment of the present application;

[0060] Figure 9 Figure 3 is a schematic diagram of a sending end device structure according to an embodiment of the present application;

[0061] Figure 10 Figure 4 is a schematic diagram of a receiving end device structure according to an embodiment of the present application;

[0062] Figure 11 Figure 5 is a schematic diagram of a data processing system structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] The present application will be further described below in detail with reference to the accompanying drawings and specific embodiments.

[0064] In the related art, for a full-amount data synchronization request of a data receiving party (containing data of a required node and corresponding sub-nodes), a data sending party traverses tree structure data of the data sending party layer by layer from the required node, and synchronizes the traversed data to the data receiving party. In addition, when there is a node data change (addition, modification or deletion) in the tree structure data of the data sending party, the data sending party synchronizes the changed node data to the receiving party, so that the data receiving party can realize data sharing.

[0065] In the above process, the full-amount data synchronization request corresponds to a large amount of data and a long processing period, while the node data change corresponds to a small amount of data and a short processing period. When there are frequent changes in the tree structure data of the data sending party, the data received by the data receiving party may be inconsistent in time sequence due to different processing periods of the data sending party, i.e., there is an error synchronization of data, which further affects the accuracy of the tree structure data of the data receiving party and produces a large amount of dirty data fragments.

[0066] Based on this, in various embodiments of the present application, after the sending end device (also referred to as a data sending party) obtains a data synchronization instruction, it processes the data synchronization instruction in order through a two-level queue according to the order of entry, and sends it to the receiving end device (also referred to as a data receiving party) in turn, so that the receiving end device can receive the data to be synchronized in turn, realize ordered synchronization of the tree structure data, and further improve the accuracy of data synchronization.

[0067] An embodiment of the present application provides a data processing method applied to a sending end device, as shown in FIG. 6, the method comprises the following steps. Figure 1

[0068] ​Step 101: obtaining at least one instruction;

[0069] Step 102: putting the obtained at least one instruction into a first queue;

[0070] Step 103: reading one instruction from the first queue in the order of entering the queue, obtaining data corresponding to the read instruction, and putting the corresponding instruction and the corresponding data into a second queue; wherein the data corresponding to one instruction is put into the second queue before the next instruction is read from the first queue;

[0071] Step 104: synchronizing the corresponding instruction and the corresponding data in the second queue to a receiving end device in the order of entering the queue.

[0072] The instruction is used to instruct the receiving end device to send data for synchronization; the at least one instruction includes a first type of instruction and / or a second type of instruction, the first type of instruction is used to subscribe to data of a specific node in a tree structure, and the second type of instruction indicates data whose synchronization increment changes.

[0073] In actual application, in step 101, the sending end device can obtain, from the receiving end device, an instruction for subscribing to data of a specific node in tree structure data stored on the sending end device side in a tree structure, that is, a first type of instruction.

[0074] The first type of instruction carries an instruction identifier, and the instruction identifier is used to indicate the type of instruction.

[0075] Specifically, when the receiving end device needs to synchronize tree structure data according to a business scenario, the receiving end device can send a subscription request carrying specific node information to the sending end device through a Hyper Text Transfer Protocol (HTTP) to synchronize data of the specific node and corresponding child nodes, so as to meet data sharing in the business scenario. The sending end device can obtain a first type of instruction based on the received subscription request.

[0076] In addition, in step 101, the sending end device can also generate an instruction for synchronizing changed data according to changed data in the tree structure data, that is, a second type of instruction; wherein the second type of instruction carries data for synchronization; the change type of the data can include addition, modification, deletion, etc., which are not limited by the embodiments of the present application.

[0077] Here, the second type of instruction carries an instruction identifier, and the instruction identifier is used to indicate the type of instruction.

[0078] Exemplarily, the sending-end device can generate a second-type instruction for synchronizing the newly-added data according to the newly-added data in the tree-structure data; the sending-end device can also generate a second-type instruction for synchronizing the modified data according to the modified data in the tree-structure data; and the sending-end device can also generate a second-type instruction for synchronizing the deleted data according to the deleted data in the tree-structure data.

[0079] Here, in actual application, in the scenario where the data in the tree-structure data changes, the amount of the changed data is usually small. Therefore, when the sending-end device generates the second-type instruction, the instruction also contains the data for synchronization.

[0080] In actual application, after the sending-end device obtains the first-type and / or second-type instruction, since the amount of data corresponding to the first-type instruction is usually large, directly responding to the instruction and obtaining the corresponding data can cause a long processing period. In the above processing, if the sending-end device obtains another second-type instruction, since the amount of data corresponding to the second-type instruction is usually small and the processing period is short, the sending-end device can preferentially synchronize the data corresponding to the second-type instruction. In this way, the time sequence of the data received by the receiving-end device can be inconsistent with that of the sending-end device, affecting the accuracy of data synchronization.

[0081] Therefore, in step 102, after the sending-end device obtains the first-type and / or second-type instruction, the sending-end device can first put the obtained instruction into a first queue.

[0082] The first queue can be understood as a message queue (MQ, Message Queue), and can specifically include types such as RabbitMQ, RocketMQ, and ActiveMQ. The type of the first queue is not limited in the embodiments of the present application.

[0083] Here, the sending-end device can put the instructions into the first queue in the order of the time when the instructions are obtained, so as to guarantee the time sequence of the data synchronization instructions.

[0084] Then, in step 103, when the sending-end device reads an instruction from the first queue in the order of entering the queue, since the first queue contains instructions of different types, the sending-end device can respond to the read instruction according to the type of the instruction, so as to perform different operations.

[0085] Based on this, in an embodiment, the type of the instruction read from the first queue is determined.

[0086] When the type of the read instruction comprises the first type, the full data under the subscription node is acquired by using the tree structure data, the first information corresponding to the read instruction and the second information and the full data under the subscription node are put into the second queue, the first information indicates the start of the full data under the subscription node, and the second information indicates the end of the full data under the subscription node.

[0087] When the type of the read instruction comprises the second type, the data corresponding to the read instruction is acquired from the first queue.

[0088] Here, in actual application, the sending end device can determine the type of the instruction according to the instruction identifier carried by the read instruction.

[0089] In actual application, when it is determined that the type of the read instruction comprises the first type, the sending end device can acquire the full data under the subscription node from the corresponding tree structure data according to the information of the subscription node carried by the instruction; the full data comprises the data of the subscription node and all child nodes of the subscription node.

[0090] Exemplarily, the sending end device can acquire the full data under the subscription node by using the tree structure data to perform traversal from the subscription node to lower layers.

[0091] Then, the sending end device puts the full data corresponding to the read instruction into the second queue, so that the receiving end device receives the data in the second queue, to realize the synchronous update of the tree structure data; wherein the instruction further comprises path attribute information of the subscription node and all child nodes, i.e., the mapping relationship between parent nodes and child nodes.

[0092] Here, the second queue can be understood as a message queue (MQ, Message Queue), and specifically can comprise types such as RabbitMQ, RocketMQ and ActiveMQ; the type of the second queue is not limited in the embodiment of the application.

[0093] In actual application, before the sending end device puts the full data corresponding to the read instruction into the second queue, the sending end device can put the first information into the second queue, the first information indicating the start of the full data under the subscription node. In addition, after the sending end device puts the full data corresponding to the read instruction into the second queue, the sending end device can further put the second information into the second queue, the second information indicating the end of the full data under the subscription node.

[0094] Here, in actual application, when the sending end device reads the instruction from the first queue, in order to guarantee the time sequence of the instruction, the sending end device needs to read the next instruction from the first queue for response after responding to the instruction and putting the data corresponding to the instruction into the second queue.

[0095] Correspondingly, when it is determined that the type of the read instruction comprises the second type, since the instruction of the second type carries corresponding data, the sending end device can directly acquire the read instruction and the corresponding data from the first queue and put them into the second queue.

[0096] In actual application, in step 104, the sending end device can synchronize the instructions and the corresponding data in the second queue to the receiving end device according to the enqueuing order, so that the receiving end device can implement synchronization of the tree structure data based on the corresponding instructions and the corresponding data.

[0097] Here, in the process of synchronization of the sending end device to the receiving end device, a synchronization failure strategy can be set to avoid the situation of synchronization failure caused by temporary abnormal conditions such as network jitter or system restart of the receiving end device, thereby improving the success rate of data synchronization.

[0098] Specifically, the sending end device can pre-set a synchronization times threshold, and synchronize the instructions and the corresponding data in the second queue to the receiving end device in turn. When the receiving end device receives the instructions and the corresponding data sent by the sending end device, the receiving end device sends confirmation information to the receiving end device to determine that the synchronization is successful. If the receiving end device does not receive the confirmation information sent by the receiving end device within a time threshold, it is determined that the synchronization is failed, and the failure times are accumulated. When the failure times of the sending end device to the receiving end device reach the synchronization times threshold, the instructions and the corresponding data can be discarded until the system of the receiving end device recovers to normal, and the receiving end device can automatically send a re-subscription request to the sending end device. In addition, the sending end device can also send a re-subscription request in the manner of manual intervention to implement synchronization of the instructions and the corresponding data.

[0099] In actual application, in the process of synchronization of the tree structure data, the receiving end device can check the received instructions and the corresponding data to guarantee the legality of the instructions. When the received instructions are illegal, the receiving end device sends a subscription reload instruction to the sending end device to re-subscribe.

[0100] Based on this, in an embodiment, the method further comprises:

[0101] acquiring the subscription reload instruction sent by the receiving end device, the subscription reload instruction being used to re-subscribe data of a specific node in the tree structure;

[0102] putting the subscription reload instruction into the first queue.

[0103] Here, after the sending end device receives the subscription reload instruction sent by the receiving end device, the sending end device can put the subscription reload instruction into the first queue according to the time sequence corresponding to the received instruction, so as to re-subscribe data of a specific node in the tree structure data.

[0104] In the above process, the two-stage queue is used to process the instructions in order, which can ensure the accuracy of data synchronization at the receiving end device.

[0105] Correspondingly, the embodiment of the application further provides a data processing method applied to a receiving end device, such as Figure 2 As shown in the figure, the method comprises:

[0106] Step 201: receiving at least one instruction and corresponding data sent by a sending end device in sequence;

[0107] Step 202: determining the legality of each instruction in the at least one instruction, responding to the corresponding instruction in the case that the corresponding instruction is legal, and storing the corresponding data in a tree structure; in the case that the corresponding instruction is not legal, buffering the corresponding instruction and the corresponding data.

[0108] The instruction is used to indicate that the data for synchronization is sent to the receiving end device; the at least one instruction comprises a first type of instruction and / or a second type of instruction, the first type of instruction is used to subscribe to the data of a specific node in the tree structure, and the second type of instruction indicates the data whose synchronization increment changes.

[0109] In actual application, in step 201, the receiving end device can receive the instructions and the corresponding data sent by the sending end device in sequence through HTTP, and the embodiment of the application does not limit the way of receiving the instructions and the data.

[0110] Here, for each received instruction, the receiving end device needs to determine the legality of the corresponding instruction before storing the corresponding data, so as to avoid the problem of dirty data fragments caused by storing the data corresponding to the illegal instruction.

[0111] Based on this, in an embodiment, the determination of the legality of the corresponding instruction comprises:

[0112] verifying the node corresponding to the corresponding instruction;

[0113] When the verification is successful, it is determined that the corresponding instruction is legal;

[0114] When the verification fails, it is determined that the corresponding instruction is not legal.

[0115] Here, since the number of nodes corresponding to the first type of instruction is usually large, and the number of nodes corresponding to the second type of instruction is usually small, the receiving end device can judge the type of the instruction before verifying the node corresponding to the corresponding instruction, so as to perform different verification operations.

[0116] Based on this, in an embodiment, the method further comprises:

[0117] determining whether the received data contains the first information, and determining that the type of the corresponding instruction contains the first type when it is determined that the received data contains the first information; the first information indicating the start of full data under the subscription node.

[0118] In actual application, for the instruction of the first type, the sending end device will put the first information and the second information corresponding to the instruction into the second queue to indicate the start and end of full data under the subscription node. Therefore, when the receiving end device receives the data synchronized from the second queue by the sending end device in sequence, when it is determined that the received data contains the first information, it can be determined that the type of the instruction contains the first type.

[0119] Based on this, in an embodiment, the method further comprises:

[0120] determining whether the received data contains the first information, and determining that the type of the corresponding instruction contains the second type when it is determined that the received data does not contain the first information; the first information indicating the start of full data under the subscription node.

[0121] Here, for the instruction of the second type, the sending end device will read the data corresponding to the instruction from the first queue and directly put the instruction and the corresponding data into the second queue. Therefore, when the receiving end device receives the data synchronized from the second queue by the sending end device in sequence, when it is determined that the received data does not contain the first information, it can be determined that the type of the instruction contains the second type.

[0122] In actual application, when the receiving end device determines that the type of the instruction contains the first type, the node corresponding to the instruction of the first type can be sequentially verified, and the data corresponding to the node that fails the verification can be cached to avoid storing dirty data fragments; wherein the instruction of the first type contains path attribute information of all nodes.

[0123] In actual application, in the process of verifying the subscription node and all child nodes corresponding to the instruction of the first type by the receiving end device, if the receiving end device can determine that the subscription node and all child nodes exist in the subscription root node or the corresponding parent node in the tree structure data on the receiving end side according to the path attribute information of the subscription node and all child nodes carried by the corresponding instruction, i.e. the mapping relationship between parent and child nodes, it can be considered that the subscription node and all child nodes pass the verification, and then it can be determined that the instruction is legal and the data corresponding to the instruction is stored to the tree structure data.

[0124] If the receiving device can determine, based on the path attribute information of the subscribed node and all child nodes carried by the corresponding instruction, that at least one of the subscribed node and all child nodes does not have a subscribed root node and corresponding parent node in the tree structure data on the receiving side, then the verification of the at least one node is considered to have failed, and the instruction and the corresponding data are cached.

[0125] In the above process, for nodes that fail verification, the receiving device can also determine the first non-existent node corresponding to the path attribute information in the tree structure data based on the path attribute information of the node that failed verification, and then determine the identifier information of the cached node. Specifically, if the receiving device cannot find the parent node of the node that failed verification in the tree structure data based on the path attribute information of the node that failed verification, the first non-existent node can be understood as the parent node of the node that failed verification, and the receiving device can cache the identifier information of the parent node of the node that failed verification. Then, the receiving device can initiate a subscription reload instruction based on all the cached first non-existent nodes.

[0126] For example, such as Figure 3 As shown, assume the subscription node corresponding to the first type of instruction is A, and its child nodes include B, C, D, E, F, G, H, and I. For subscription node A, since node A is the root subscription node corresponding to the instruction, the receiving device can determine that node A's verification was successful and will store node A's identification information and corresponding data in the tree structure data.

[0127] For node B, the receiving device can determine that node A is the parent node of node B based on path attribute information, and can find the parent node A in the tree structure data. Therefore, the receiving device can determine that node B has been successfully verified and store the identification information and corresponding data of node B in the tree structure data. Similarly, for nodes D and E, the receiving device can determine that node B is the parent node of nodes D and E, and can find the parent node B in the tree structure data. Therefore, the receiving device can also store the identification information and corresponding data of nodes D and E in the tree structure data.

[0128] For node C, the receiving end device can find the parent node A corresponding to node C in the tree structure data to determine that node C passes the verification, and synchronizes node C and the corresponding data to the tree structure data. Assuming that node C and the corresponding data are lost in the synchronization process, for node F, since the receiving end device cannot find the parent node C corresponding to node F in the tree structure data based on the path attribute information, and node F is not the subscription root node, the receiving end device can determine that node F fails the verification, and queries upward to obtain the first non-existent node C in the tree structure data according to the path attribute information of node F. Then, the receiving end device can cache the identification information of node C.

[0129] Correspondingly, for nodes G and H, the receiving end device cannot find the corresponding parent node C in the tree structure data, and nodes G and H are not the subscription root nodes, so it can be determined that nodes G and H fail the verification. Then, the receiving end device can query the first non-existent node C corresponding to nodes G and H in the tree structure data based on the path attribute information of nodes G and H. Since the receiving end device has cached the identification information of node C, it is not necessary to cache again.

[0130] In addition, for node I, based on the path attribute information, the receiving end device cannot find the corresponding parent node G in the tree structure data, and node I is not the subscription root node, so it can be determined that node I fails the verification. The receiving end device can query the first non-existent node C corresponding to node I. Since the receiving end device has cached the identification information of node C, it is not necessary to cache again.

[0131] In this way, it can be avoided that the receiving end device stores dirty data fragments (i.e., data corresponding to illegal nodes) into the tree structure data, affecting the accuracy of the tree structure data on the receiving end side.

[0132] In actual application, in order to guarantee the integrity of the tree structure data on the receiving end side, for the illegal first type of instruction, the receiving end device can initiate a re-subscription request to the sending end device to synchronize the data corresponding to the illegal first type of instruction to the tree structure data.

[0133] Here, since the data corresponding to the first type of instruction is usually large in amount, the number of nodes that fail the verification can be large, in order to avoid repeatedly sending the subscription overload instruction, the receiving end device can check all nodes of the first type of instruction, and then send the subscription overload instruction based on the verification results of all nodes.

[0134] Based on this, in an embodiment, the type of the corresponding instruction includes a first type, and the method further includes:

[0135] determine and cache all node related information of the corresponding instruction corresponding to the check failure in the case that the corresponding instruction is illegal;

[0136] initiate a subscription reload instruction to the sending end device based on the cached all node related information; the subscription reload instruction is used to resubscribe the data of the specific node in the tree structure.

[0137] The node related information can include identification information of the node.

[0138] In actual application, the receiving end device can cache all node related information of the check failure in the process of checking the first type of instruction by the receiving end device, and then send the subscription reload instruction after checking all nodes corresponding to the first type of instruction.

[0139] In an embodiment,

[0140] receive second information; the second information indicates the end of the full amount of data under the subscription node;

[0141] initiate a subscription reload instruction to the sending end device based on the cached all node related information.

[0142] Here, since the first type of instruction corresponds to the data containing the second information, the second information indicates the end of the full amount of data under the subscription node, therefore, the receiving end device can determine that all data corresponding to the first type of instruction has been received and checked after receiving the second information. Then the receiving end device can send the subscription reload instruction based on the related information of all nodes of the check failure.

[0143] Here, since the receiving end device sends the subscription reload instruction based on the related information of all nodes of the check failure, the parent-child relationship between all nodes of the check failure can be determined in this process. In this way, for the nodes of the check failure with the parent-child relationship, the data synchronization request of the parent node and the corresponding child node can be realized at the same time through one subscription reload instruction, which reduces the number of subscription reload instructions and avoids the problem of large-scale repeated initiation of the subscription reload instruction which causes the sending end device to have a large processing pressure.

[0144] Exemplarily, the receiving end device can send the subscription reload instruction to the sending end device through the HTTP mode, and the application embodiment does not limit the mode of initiating the subscription reload instruction.

[0145] In actual application, for the second type of instruction, since the number of nodes and the amount of data corresponding to the second type of instruction are small, the sending end device can perform the corresponding check operation according to the check result of each node; wherein the second type of instruction contains the path attribute information of the corresponding node.

[0146] Based on this, in an embodiment, the type of the corresponding instruction includes a second type; the method further includes:

[0147] In the case where the corresponding instruction is illegal, determining first node related information corresponding to a check failure of the corresponding instruction;

[0148] Based on the first node related information, initiating the subscription reloading instruction to the sending end device; the subscription reloading instruction is used to resubscribe data of a specific node in a tree structure.

[0149] The first node related information can include first node identification information.

[0150] In actual application, the receiving end device can query a first non-existent node corresponding to path attribute information of the first node of the check failure in the tree structure data according to the path attribute information of the first node of the check failure, and initiate the subscription reloading instruction to the sending end device based on identification information of the first non-existent node; the subscription reloading instruction carries identification information of the child node.

[0151] For the first type of instruction and the second type of instruction, the receiving end device initiates the subscription reloading instruction based on the node of the check failure, which can realize automatic repair of the tree structure data on the receiving end side.

[0152] Embodiments of the present application also provide a data processing method, as shown in Figure 4 The method includes:

[0153] Step 401: A sending end device acquires at least one instruction;

[0154] The instruction is used to instruct sending of data for synchronization to a receiving end device; the at least one instruction includes a first type of instruction and / or a second type of instruction, the first type of instruction is used to subscribe data of a specific node in a tree structure, and the second type of instruction instructs data of which a synchronization increment has changed;

[0155] Step 402: The sending end device puts the acquired at least one instruction into a first queue;

[0156] Step 403: An instruction is read from the first queue in an order of entering the queue, corresponding data of the read instruction is acquired, and the corresponding instruction and the corresponding data are put into a second queue;

[0157] The corresponding data of an instruction is put into the second queue, and then a next instruction is read from the first queue;

[0158] Step 404: The sending end device synchronizes the corresponding instruction and the corresponding data in the second queue to the receiving end device in the order of entering the queue.

[0159] Step 405: The receiving device receives at least one instruction and corresponding data sequentially sent by the sending device;

[0160] Step 406: For each instruction in at least one instruction, the receiving device determines the validity of the corresponding instruction. If the corresponding instruction is valid, the receiving device responds to the corresponding instruction and stores the corresponding data in a tree structure.

[0161] Step 407: If the corresponding instruction is invalid, the receiving device caches the corresponding instruction and the corresponding data.

[0162] It should be noted that the specific processing procedures of the sending and receiving devices have been detailed above and will not be repeated here.

[0163] This application provides a data processing method in which a sending device acquires at least one instruction, which instructs the sending of data for synchronization to a receiving device; the acquired at least one instruction is placed into a first queue; an instruction is read from the first queue in enqueue order, and in response to the read instruction, the corresponding data is acquired and placed into a second queue; the corresponding instructions and corresponding data in the second queue are synchronized to the receiving device in enqueue order; the receiving device receives at least one instruction and corresponding data sent sequentially by the sending device; for each of the at least one instructions, the validity of the corresponding instruction is determined; if the corresponding instruction is valid, the receiving device responds to the corresponding instruction and stores the corresponding data in a tree structure; if the corresponding instruction is invalid, the receiving device caches the corresponding instruction and corresponding data. The solution provided in this application allows the sending device to process the instructions for data synchronization in an orderly manner through the first and second queues and send them to the receiving device, enabling the receiving device to receive synchronized data sequentially and store it in a tree structure. This ensures the timing of tree structure data synchronization, thereby improving the accuracy of the tree structure data on the receiving end.

[0164] The present application will be further described in detail below with reference to application examples.

[0165] like Figure 5 As shown, the tree structure data synchronization system framework of this application embodiment consists of a tree structure data provider (i.e., the sending device) and a tree structure data receiver (i.e., the receiving device), used to ensure the orderly synchronization of data between the tree structure data provider and the tree structure data receiver. Specifically, the tree structure data provider may include:

[0166] The tree structure data provider service unit is used to receive operation instructions for tree structure data, perform traversal retrieval of tree structure data, and send information such as subscription processing start (i.e., first information) and subscription processing end (i.e., second information);

[0167] Tree structure data provider data storage unit, used to store tree structure data entities;

[0168] The first-level queue unit (i.e., the first queue) is used to store operation instructions for tree structure data according to the time sequence. It mainly includes full operation instructions for subscribing to tree structure data (i.e., first type of instructions) and incremental operation instructions for adding, deleting, and modifying tree structure data (i.e., second type of instructions).

[0169] The queue transfer unit is used to retrieve operation instructions from the primary queue unit according to the enqueue order and to process different types of operation instructions accordingly.

[0170] The second-level queue unit (i.e., the second queue) is used to store the processed operation instructions in the first-level queue unit according to the time sequence. It mainly includes instructions for starting subscription processing, adding all data, ending subscription processing, as well as incremental operation instructions such as adding, deleting, and modifying tree structure data.

[0171] The tree structure data provider's three-party interface unit is used to retrieve operation instructions from the secondary queue unit according to the enqueue order and synchronize the retrieved operation instructions to the tree structure data receiver.

[0172] Specifically, the tree structure data receiver includes:

[0173] The tree structure data receiver service unit is used to receive operation instructions sent by the three-party interface unit of the tree structure data provider, and to judge the legality of the data corresponding to the operation instructions.

[0174] The tree structure data receiver data storage unit is used to store the received valid tree structure data;

[0175] The tree structure data receiver retry cache data unit is used to store invalid nodes and data in the data corresponding to the operation instructions, so as to realize automatic repair in the case of abnormal tree structure data.

[0176] like Figure 6 As shown, the process of the tree structure data provider receiving an operation instruction and synchronizing data with the tree structure data receiver in the tree structure data synchronization system includes the following steps:

[0177] Step 601: When the tree structure data provider service unit in the tree structure data provider determines that data addition, modification, or deletion has occurred in the tree structure data, it generates the corresponding incremental operation instruction and sends it to the first-level queue unit;

[0178] The incremental operation instruction carries a node identifier and corresponding data.

[0179] Here, in actual application, when the tree structure data provided by the tree structure data provider side is subjected to data addition, modification, deletion, or other operations, corresponding log information is generated. The tree structure data provider service unit can determine whether the tree structure data has changed based on the log information.

[0180] In actual application, the tree structure data provider service unit can send the corresponding incremental operation instruction to the primary queue unit through MQTT, XMPP, Stomp, AMQP, OpenWire, or other message protocols.

[0181] Step 602: The tree structure data receiver service unit of the tree structure data receiver sends a subscription full data request to the tree structure data provider service unit.

[0182] The subscription full data request carries subscription node information.

[0183] Step 603: After receiving the subscription full data request sent by the tree structure data receiver service unit, the tree structure data provider service unit generates a corresponding subscription full operation instruction and sends it to the primary queue unit.

[0184] In steps 601 and 603, the tree structure data provider service unit sends the corresponding operation instruction to the primary queue unit according to the order in which the operation instruction is generated.

[0185] It should be noted that the execution order of steps 601 and 603 is not sequential.

[0186] Step 604: The primary queue unit reads the operation instruction and sends it to the queue transfer unit.

[0187] Here, the primary queue unit reads one operation instruction in the order in which the operation instruction is received and sends it to the queue transfer unit, and then step 605 is executed.

[0188] Step 605: The queue transfer unit determines whether the read operation instruction is a subscription full operation instruction.

[0189] If the operation instruction is a subscription full operation instruction, step 606 is executed; if the operation instruction is not a subscription full operation instruction, i.e., the operation instruction is an incremental operation instruction, the queue transfer unit directly sends the read operation instruction to the secondary queue unit and executes step 610.

[0190] Step 606: The queue transfer unit sends the read operation instruction to the tree structure data provider service unit.

[0191] Here, when the queuing unit determines that the operation instruction is a subscription full operation instruction, the queuing unit sends the subscription full operation instruction to the tree structure data provider service unit in a remote procedure call (RPC) manner, so that the tree structure data provider service unit performs step 607.

[0192] Step 607: The tree structure data provider service unit sends subscription processing start information to the secondary queue unit.

[0193] In actual application, after receiving the subscription full operation instruction, the tree structure data provider service unit sends subscription processing start information to the secondary queue unit, so as to determine the type of the corresponding instruction when the tree structure data provider service unit receives the corresponding instruction. Then, step 608 is performed.

[0194] Step 608: The tree structure data provider service unit traverses the data corresponding to the subscription node and sends the data of the subscription node obtained by traversal to the secondary queue unit.

[0195] In actual application, the tree structure data provider service unit traverses each layer of node information from the subscription node to the lower layer in sequence, and sends a subscription node addition instruction to the secondary queue unit; the subscription node addition instruction carries the node identifier and data of the subscription node and the corresponding child node.

[0196] At the same time, when the tree structure data provider service unit traverses from the subscription node, the node path attribute information (i.e., path attribute information) of each node is also obtained. That is, the subscription node addition instruction also carries the node path attribute information of each node in the data.

[0197] Step 609: After traversal, the tree structure data provider service unit sends subscription processing end information to the secondary queue unit.

[0198] In actual application, the tree structure data provider service unit also sends subscription processing end information to the primary queue unit, so that the primary queue unit reads the next operation instruction and sends it to the queuing unit.

[0199] Step 610: The secondary queue unit receives the related operation instruction.

[0200] Step 611: The secondary queue unit synchronizes the operation instruction to the tree structure data receiver in the order of entering the queue.

[0201] Specifically, the tree structure data provider three-party connection unit obtains the operation instruction from the secondary queue unit in the order of entering the queue, and synchronizes the operation instruction to the tree structure data receiver service unit in an HTTP manner.

[0202] Step 612: The tree structure data provider three-party joint unit judges whether the operation instruction is synchronized to the tree structure data receiver service unit within N times.

[0203] The value of N is an integer greater than or equal to 1, such as 3.

[0204] In the process of synchronizing the operation instruction from the tree structure data provider three-party joint unit to the tree structure data receiver service unit, if the tree structure data receiver service unit receives the operation instruction, it will send receiving information to the tree structure data provider three-party joint unit to inform the tree structure data provider three-party joint unit of the synchronization success; otherwise, it will not send receiving information to the tree structure data provider three-party joint unit.

[0205] When the tree structure data provider three-party joint unit does not receive the receiving information sent by the tree structure data receiver service unit within the preset time threshold, it is determined that this synchronization fails, and the synchronization operation is reinitiated.

[0206] If it is determined that the operation instruction is successfully synchronized to the tree structure data receiver service unit within N times (such as 3 times), step 614 is performed; otherwise, step 613 is performed.

[0207] Step 613: The tree structure data provider three-party joint unit discards the operation instruction and the corresponding data.

[0208] Here, if the tree structure data provider three-party joint unit cannot synchronize the operation instruction and the corresponding data to the tree structure data receiver service unit within N times (such as 3 times), it means that the tree structure data receiver system may have a fault. At this time, the operation instruction and the corresponding data are discarded, and the data is repaired by automatically resending the subscription instruction or manually initiating data reloading after the tree structure data receiver system is restarted.

[0209] Step 614: The tree structure data receiver service unit checks the received operation instruction and the corresponding data.

[0210] Step 615: The tree structure data receiver service unit judges whether the received operation instruction is a subscription node addition instruction.

[0211] Here, if the operation instruction contains subscription processing start information, it is determined to be a subscription node addition instruction, and step 616 is performed; if the operation instruction does not contain subscription processing start information, it is determined to be an incremental operation instruction, and step 621 is performed.

[0212] Step 616: The tree structure data receiver service unit checks all nodes in the received data.

[0213] Here, the tree structure data receiver service unit can check all nodes in the received data according to the node identifier and node path attribute information carried by the subscription node addition instruction.

[0214] Step 617: The tree structure data receiver service unit determines whether the current node is a subscription root node.

[0215] In actual application, the tree structure data receiver service unit determines whether the current node is a subscription root node by matching the node identifier carried by the subscription node addition instruction with the subscription node information carried by the sent subscription full data request.

[0216] If the two match, the tree structure data receiver service unit determines that the current node is the subscription root node corresponding to the subscription full data request, and then step 623 is performed; otherwise, step 618 is performed.

[0217] Step 618: The tree structure data receiver service unit determines whether the current node has a parent node.

[0218] If the tree structure data receiver service unit determines that the node has a parent node according to the node path attribute of the node in the received data, step 623 is performed; otherwise, step 619 is performed.

[0219] Step 619: The tree structure data receiver service unit caches the node identifier.

[0220] In actual application, when the tree structure data receiver service unit determines that the current node has no parent node and is not a subscription root node, it can be determined that the current node is an illegal node. In this case, the tree structure data receiver service unit will not store the data corresponding to the current node to the tree structure data receiver data storage unit, but will determine the first non-existent node corresponding to the node path attribute in the tree structure data according to the node path attribute of the current node, and cache the node identifier of the first non-existent node to the tree structure data receiver retry cache data unit, so that the tree structure data receiver service unit initiates a reload subscription request based on the cached node identifier.

[0221] In this way, it can prevent the loss of part of the data in abnormal conditions, and continue to synchronize the data after the abnormal recovery to produce data suspension, thereby avoiding the problem of dirty data fragments caused by illegal nodes into the warehouse, and at the same time preparing for subsequent data recovery.

[0222] Step 620: After receiving the subscription processing end signal, the tree structure data receiver service unit initiates a reload subscription request based on all cached nodes.

[0223] Here, since the number of nodes and the amount of data corresponding to the subscription node addition instruction are large, in order to avoid repeated sending of the heavy subscription request, the tree structure data receiver service unit cyclically executes steps 617-619 to check all nodes in the received data, and sends the heavy subscription request based on the checking results of all nodes.

[0224] When the subscription processing end signal is received, the tree structure data receiver service unit can determine that all nodes in the received data have been checked, and obtain all cached node identifiers from the tree structure data receiver retry cache data unit, and sequentially initiate a heavy subscription request to the tree structure data provider service unit based on the cached node identifiers.

[0225] Step 621: The tree structure data receiver service unit determines whether the current node has a parent node.

[0226] In actual application, when the operation instruction is an incremental operation instruction, and the tree structure data receiver service unit determines that the node has a parent node according to the node path attribute of the node in the received data, step 623 is executed; otherwise, step 622 is executed.

[0227] Step 622: The tree structure data receiver service unit initiates a re-subscription request based on the current node.

[0228] Here, since the amount of data corresponding to the incremental operation instruction is small, the tree structure data receiver service unit can directly initiate a re-subscription request to the tree structure data provider service unit according to the current node.

[0229] Step 623: The tree structure data receiver service unit updates the data that passes the check into the database.

[0230] In actual application, when the tree structure data receiver service unit determines that the node in the received data is a legal node, the received data can be synchronously updated to the tree structure data receiver data storage unit.

[0231] In the application embodiment, a two-level queue mode is proposed to perform data synchronization, and the timing of data synchronization is realized. The first-level queue is used to store the tree structure data subscription full data and the tree structure data addition, deletion and modification business operation instructions, and does not directly contain all data operations. The second-level queue is used to store the data operation instructions for synchronization and auxiliary data repair signal data, mainly including the tree structure data addition, deletion and modification operation instructions and the subscription processing start and completion signal instructions. Through the design of the two-level queue, not only can the pressure of traversing data on the tree structure data provider brought by a large number of subscription full data operation instructions be reduced, but also the subscription full data operation and the incremental data update operation can be separated, thereby reducing the data recovery cost of the tree structure data receiver caused by the difficulty of the tree structure data provider to guarantee the timing of the synchronization data

[0232] Secondly, the tree structure data provider synchronously transmits the data operation instruction to the tree structure data receiver after receiving the data operation instruction. When the data synchronization fails, within a certain threshold range, the tree structure data provider can initiate re-synchronization of the data that fails to synchronize based on a retry synchronization strategy, avoid the situation that the synchronization fails due to network jitter or system exception of the tree structure data receiver, guarantee the stability of data synchronization in an abnormal scenario, and effectively prevent the generation of a large number of dirty data fragments in the database.

[0233] In addition, in the application embodiment, the tree structure data receiver verifies the legality of the data corresponding to the data operation instruction after receiving the data operation instruction transmitted by the tree structure data provider, and initiates subscription reloading for the illegal node through retry caching, to realize automatic repair of the data. Based on the above automatic repair mechanism, after a data change operation triggers a repair condition for a period of time, the tree structure data receiver data and the tree structure data provider data can be made consistent. At the same time, the tree structure data receiver can also receive manual intervention to initiate subscription reloading of related nodes to realize repair of local tree structure data. In this way, the accuracy and integrity of the receiver side tree structure data can be guaranteed while greatly reducing manual verification work.

[0234] In order to realize the scheme of the sending end device side of the application embodiment, the application embodiment further provides a data processing apparatus arranged on a sending end device, as shown in Figure 7 The apparatus comprises:

[0235] An acquisition unit 701 is configured to acquire at least one instruction, the instruction being used to instruct to send data for synchronization to a receiving end device; the at least one instruction comprises a first type of instruction and / or a second type of instruction, the first type of instruction being used to subscribe to data of a specific node in a tree structure; the second type of instruction instructs to synchronize data in which an increment has changed.

[0236] The first processing unit 702 is configured to put the obtained at least one instruction into a first queue.

[0237] The second processing unit 703 is configured to read one instruction from the first queue in the order of entering the queue, obtain data corresponding to the read instruction, and put the read instruction and the corresponding data into a second queue; wherein the data corresponding to one instruction is put into the second queue before the next instruction is read from the first queue.

[0238] The synchronization unit 704 is configured to synchronize the read instruction and the corresponding data in the second queue to a receiving end device in the order of entering the queue.

[0239] In an embodiment, the second processing unit 703 is configured to:

[0240] determine the type of the read instruction from the first queue;

[0241] when the type of the read instruction contains a first type, obtain full data under a subscription node by using tree structure data, and put the read instruction, first information and second information corresponding to the read instruction, and the full data under the subscription node into the second queue, the first information indicating the start of the full data under the subscription node, and the second information indicating the end of the full data under the subscription node;

[0242] when the type of the read instruction contains a second type, obtain the data corresponding to the read instruction from the first queue.

[0243] In an embodiment, the obtaining unit 701 is further configured to:

[0244] obtain a subscription reloading instruction sent by the receiving end device, the subscription reloading instruction being used to resubscribe data of a specific node in the tree structure;

[0245] The first processing unit 702 is further configured to:

[0246] put the subscription reloading instruction into the first queue.

[0247] It should be noted that the obtaining unit 701 and the first processing unit 702 are equivalent to the function of the tree structure data provider service unit in the application embodiment; the function of the second processing unit 703 is equivalent to the function of the queue transfer unit in the application embodiment; and the function of the synchronization unit 704 is equivalent to the function of the tree structure data provider three-party connection unit in the application embodiment.

[0248] In actual application, the acquisition unit 701 and the synchronization unit 704 can be implemented by a communication interface in a data processing device in combination with a processor; and the first processing unit 702 and the second processing unit 703 can be implemented by a processor in the data processing device.

[0249] To implement the scheme at the receiving end device side of the embodiments of the present application, the embodiments of the present application further provide a data processing device arranged on a receiving end device, as shown in the figure, the device comprises: Figure 8

[0250] a receiving unit 801, configured to receive at least one instruction and corresponding data sent by a sending end device in sequence; the instruction is used to instruct to send data for synchronization to the receiving end device; the at least one instruction comprises a first type of instruction and / or a second type of instruction; the first type of instruction is used to subscribe to data of a specific node in a tree structure; the second type of instruction indicates data of which a synchronization increment changes;

[0251] a third processing unit 802, configured to determine the legality of each instruction in the at least one instruction, and in the case that the corresponding instruction is legal, store the corresponding data in a tree structure manner in response to the corresponding instruction;

[0252] a fourth processing unit 803, configured to cache the corresponding instruction and the corresponding data in the case that the corresponding instruction is not legal.

[0253] In an embodiment, the third processing unit 802 is configured to:

[0254] verify the node corresponding to the corresponding instruction;

[0255] when the verification succeeds, determine that the corresponding instruction is legal;

[0256] when the verification fails, determine that the corresponding instruction is not legal.

[0257] In an embodiment, the third processing unit 802 is further configured to:

[0258] determine whether the received data contains first information, and when it is determined that the received data contains the first information, determine that the type of the corresponding instruction contains the first type; the first information indicates the start of full data under a subscribed node.

[0259] In an embodiment, the third processing unit 802 is further configured to:

[0260] determine whether the received data contains first information, and when it is determined that the received data does not contain the first information, determine that the type of the corresponding instruction contains the second type; the first information indicates the start of full data under a subscribed node.​

[0261] In an embodiment, the type of the corresponding instruction comprises a first type, and the fourth processing unit 803 is further configured to:

[0262] In the case where the corresponding instruction is illegal, determine and cache all node-related information corresponding to a check failure of the corresponding instruction;

[0263] Based on the cached all node-related information, initiate a subscription reloading instruction to the sending end device; the subscription reloading instruction is used to resubscribe data of a specific node in the tree structure.

[0264] In an embodiment, the fourth processing unit 803 is configured to:

[0265] receive second information; the second information indicates an end of full-amount data under a subscription node;

[0266] Based on the cached all node-related information, initiate a subscription reloading instruction to the sending end device.

[0267] In an embodiment, the type of the corresponding instruction comprises a second type, and the fourth processing unit 803 is further configured to:

[0268] In the case where the corresponding instruction is illegal, determine first node-related information corresponding to a check failure of the corresponding instruction;

[0269] Based on the first node-related information, initiate the subscription reloading instruction to the sending end device; the subscription reloading instruction is used to resubscribe data of a specific node in the tree structure.

[0270] Here, it should be noted that the functions of the receiving unit 801, the third processing unit 802, and the fourth processing unit 803 are equivalent to the functions of the tree structure data receiving party service unit in the application embodiment.

[0271] In actual application, the receiving unit 801 can be implemented by a communication interface in a data processing device; the third processing unit 802 can be implemented by a processor in a fault processing device; and the fourth processing unit 803 can be implemented by a processor in combination with a communication interface in a data processing device.

[0272] It should be noted that the data processing apparatus provided in the above embodiment is only used for example to illustrate the division of the above program units, and in actual application, the above processing can be completed by different program units according to needs, that is, the internal structure of the apparatus is divided into different program units to complete all or part of the above-described processing. In addition, the data processing apparatus and the data processing method provided in the above embodiment belong to the same concept, and the specific implementation process is described in the method embodiment, which will not be repeated here.

[0273] Based on the hardware implementation of the above program modules, and in order to realize the method of the sending end device side of the embodiment of the application, the embodiment of the application further provides a sending end device, as shown in the following table, the sending end device 900 includes: Figure 9

[0274] The first communication interface 901 can interact with the receiving end device.

[0275] The first processor 902 is connected with the first communication interface 901 to realize information interaction with the receiving end device, and is used to run the computer program to execute the method provided by one or more technical solutions of the sending end device side; the computer program is stored on the first memory 903.

[0276] Specifically, the first processor 902 is configured to:

[0277] Obtain at least one instruction through the first communication interface 901, the instruction is used to indicate that the data used for synchronization is sent to the receiving end device; the at least one instruction contains a first type of instruction and / or a second type of instruction, the first type of instruction is used to subscribe the data of a specific node in the tree structure; the second type of instruction indicates the data whose synchronization increment changes;

[0278] Put the obtained at least one instruction into the first queue;

[0279] Read one instruction from the first queue in the order of entering the queue, obtain the data corresponding to the corresponding instruction in response to the read instruction, and put the corresponding instruction and the corresponding data into the second queue; wherein the data corresponding to one instruction is put into the second queue, and then the next instruction is read from the first queue;

[0280] Synchronize the corresponding instruction and the corresponding data in the second queue to the receiving end device through the first communication interface 901 in the order of entering the queue.

[0281] In an embodiment, the first processor 902 is configured to:

[0282] Determine the type of the instruction read from the first queue; ​

[0283] When the type of the read instruction comprises the first type, the full amount of data under the subscription node is acquired by using the tree structure data, and the first information and the second information corresponding to the read instruction and the full amount of data under the subscription node are put into the second queue, the first information indicating the start of the full amount of data under the subscription node, and the second information indicating the end of the full amount of data under the subscription node.

[0284] When the type of the read instruction comprises the second type, the data corresponding to the read instruction is acquired from the first queue.

[0285] In an embodiment, the first processor 902 is further configured to:

[0286] The subscription reload instruction sent by the receiving end device is acquired through the first communication interface 901, and the subscription reload instruction is used to resubscribe the data of a specific node in the tree structure.

[0287] The subscription reload instruction is put into the first queue.

[0288] It should be noted that the specific processing process of the first processor 902 can be understood with reference to the above method.

[0289] Of course, in actual application, various components in the sending end device are coupled together through the bus system 904. It can be understood that the bus system 904 is used to realize the connection and communication between the components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus and a state signal bus. However, in order to clearly illustrate, all kinds of buses are marked as the bus system 904 in the Figure 9 .

[0290] The first memory 903 in the embodiment of the application is used to store various types of data to support the operation of the sending end device 900. Examples of these data include any computer programs used to operate on the sending end device 900.

[0291] The method disclosed by the embodiments of the present application can be applied to the first processor 902 or implemented by the first processor 902. The first processor 902 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the first processor 902 or instructions in the form of software. The first processor 902 described above can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 902 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present application, the hardware decoding processor can be directly embodied to execute the steps of the method, or the hardware and software modules in the decoding processor can be combined to execute the steps of the method. The software module can be located in a storage medium, which is located in the first memory 903, and the first processor 902 reads the information in the first memory 903 to combine the hardware to complete the steps of the method.

[0292] In the exemplary embodiments, the sending end device 900 can be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, micro controllers (MCUs), microprocessors (Microprocessors), or other electronic elements, for executing the foregoing method.

[0293] Based on the hardware implementation of the above program module, and in order to implement the method on the receiving end device side according to the embodiments of the present application, the embodiments of the present application further provide a receiving end device, as shown in the following figure: Figure 10 The receiving end device 1000 includes:

[0294] The second communication interface 1001 can interact with the sending end device.

[0295] The second processor 1002 is connected with the second communication interface 1001 to realize information interaction with the sending end device, and is used for running a computer program to execute the method provided by one or more technical solutions on the receiving end device side; and the computer program is stored on the second memory 1003.

[0296] Specifically, the second processor 1002 is configured to:

[0297] receive at least one instruction and corresponding data sent by the sending end device in sequence through the second communication interface 1001; the instruction is used to instruct to send data for synchronization to the receiving end device; the at least one instruction includes a first type of instruction and / or a second type of instruction, the first type of instruction is used to subscribe to data of a specific node in a tree structure, and the second type of instruction indicates data whose synchronization increment changes;

[0298] For each instruction in the at least one instruction, determine the legality of the corresponding instruction, and in the case that the corresponding instruction is legal, respond to the corresponding instruction to store the corresponding data in a tree structure; in the case that the corresponding instruction is not legal, cache the corresponding instruction and the corresponding data.

[0299] In an embodiment, the second processor 1002 is configured to:

[0300] verify the node corresponding to the corresponding instruction;

[0301] When the verification is successful, it is determined that the corresponding instruction is legal;

[0302] When the verification fails, it is determined that the corresponding instruction is not legal.

[0303] In an embodiment, the second processor 1002 is further configured to:

[0304] determine whether the received data contains first information, and when it is determined that the received data contains the first information, determine that the type of the corresponding instruction contains the first type; the first information indicates the start of full data under a subscribed node.

[0305] In an embodiment, the second processor 1002 is further configured to:

[0306] determine whether the received data contains first information, and when it is determined that the received data does not contain the first information, determine that the type of the corresponding instruction contains the second type; the first information indicates the start of full data under a subscribed node.

[0307] In an embodiment, the type of the corresponding instruction contains the first type, and the second processor 1002 is further configured to:

[0308] determine and cache all node-related information corresponding to the check failure of the corresponding instruction in the case that the corresponding instruction is illegal;

[0309] initiate a subscription reloading instruction to the sending end device based on the cached all node-related information; the subscription reloading instruction is used to resubscribe the data of a specific node in the tree structure.

[0310] In an embodiment, the second processor 1002 is configured to:

[0311] receive second information; the second information indicates the end of full-amount data under the subscribed node;

[0312] initiate a subscription reloading instruction to the sending end device based on the cached all node-related information.

[0313] In an embodiment, the type of the corresponding instruction includes a second type, and the second processor 1002 is further configured to:

[0314] determine first node-related information corresponding to the check failure of the corresponding instruction in the case that the corresponding instruction is illegal;

[0315] initiate the subscription reloading instruction to the sending end device based on the first node-related information; the subscription reloading instruction is used to resubscribe the data of a specific node in the tree structure.

[0316] It should be noted that the specific processing process of the second processor 1002 can be understood with reference to the above method.

[0317] Of course, in actual application, various components in the receiving end device are coupled together through the bus system 1004. It can be understood that the bus system 1004 is used to realize the connection and communication between these components. In addition to including a data bus, the bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, in order to clearly illustrate, all kinds of buses are marked as the bus system 1004 in the Figure 10 .

[0318] The second memory 1003 in the embodiment of the application is used to store various types of data to support the operation of the receiving end device 1000. Examples of these data include any computer programs used to operate on the receiving end device 1000.

[0319] The method disclosed by the embodiments of the present application can be applied to the second processor 1002 or implemented by the second processor 1002. The second processor 1002 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the second processor 1002. The second processor 1002 described above can be a general processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The second processor 1002 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present application, the hardware decoding processor can be directly embodied to execute the steps, or the combination of hardware and software modules in the decoding processor can be executed. The software module can be located in the storage medium, which is located in the second storage 1003, and the second processor 1002 reads the information in the second storage 1003 to complete the steps of the above method in combination with the hardware.

[0320] In the exemplary embodiments, the receiving end device 1000 can be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general processors, controllers, MCUs, microprocessors, or other electronic elements, for executing the above method.

[0321] To implement the method provided by the embodiments of the present application, the embodiments of the present application further provide a data processing system, as shown in Figure 11 The system includes a sending end device 1101 and a receiving end device 1102.

[0322] Here, it should be noted that the specific processing process of the sending end device 1101 and the receiving end device 1102 has been described in detail above, and will not be described here.

[0323] In the exemplary embodiments, the embodiments of the present application further provide a storage medium, i.e. a computer storage medium, specifically a computer readable storage medium, for example, including a first storage 903 storing a computer program, which can be executed by a first processor 902 of a sending end device 900 to complete the steps of the above-mentioned sending end device side method, and further including a second storage 1003 storing a computer program, which can be executed by a second processor 1002 of a receiving end device 1000 to complete the steps of the above-mentioned receiving end device side method. The computer readable storage medium can be FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disc, or CD-ROM, etc.

[0324] It should be noted that "first", "second", and the like are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0325] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0326] The above is only a preferred embodiment of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized by, The method is applied to a sending terminal device, and comprises the following steps: obtaining at least one instruction, the instruction being used to indicate that data for synchronization is sent to a receiving terminal device; the at least one instruction comprises a first type of instruction and / or a second type of instruction, the first type of instruction being used to subscribe to data of a specific node in a tree structure, and the second type of instruction indicating data in which a synchronization increment changes; putting the obtained at least one instruction into a first queue; reading one instruction from the first queue in a read-in order, obtaining corresponding data of the read instruction, and putting the corresponding instruction and the corresponding data into a second queue in response to the read instruction; wherein the corresponding data of one instruction is put into the second queue, and then the next instruction is read from the first queue; synchronizing the corresponding instruction and the corresponding data in the second queue to the receiving terminal device in the read-in order; wherein the method further comprises: judging the type of the instruction read from the first queue; when the type of the read instruction comprises the first type, obtaining full data under a subscription node by using tree structure data, and putting the read instruction corresponding first information, second information and full data under the subscription node into the second queue, the first information indicating the beginning of the full data under the subscription node, and the second information indicating the end of the full data under the subscription node; when the type of the read instruction comprises the second type, obtaining the data corresponding to the read instruction from the first queue.

2. The method of claim 1, wherein, The method further comprises: obtaining a subscription reloading instruction sent by the receiving terminal device, the subscription reloading instruction being used to resubscribe to data of a specific node in a tree structure; putting the subscription reloading instruction into the first queue.

3. A data processing method, characterized by, The method is applied to a receiving terminal device, and comprises the following steps: receiving at least one instruction and corresponding data sent by a sending terminal device in sequence; the instruction is used to indicate that data for synchronization is sent to the receiving terminal device; the at least one instruction comprises a first type of instruction and / or a second type of instruction, the first type of instruction being used to subscribe to data of a specific node in a tree structure, and the second type of instruction indicating data in which a synchronization increment changes; for each instruction in the at least one instruction, determining the legality of the corresponding instruction, in the case that the corresponding instruction is legal, responding to the corresponding instruction to store the corresponding data in a tree structure mode; in the case that the corresponding instruction is not legal, buffering the corresponding instruction and the corresponding data; wherein the type of the corresponding instruction comprises the first type, in the case that the corresponding instruction is not legal, determining and buffering all node related information of the corresponding instruction in which a check fails; based on the buffered all node related information, initiating a subscription reloading instruction to the sending terminal device; the subscription reloading instruction is used to resubscribe to data of a specific node in a tree structure; or the type of the corresponding instruction comprises the second type, in the case that the corresponding instruction is not legal, determining first node related information of the corresponding instruction in which a check fails. initiate the subscription reloading instruction to the sending end device based on the first node-related information; the subscription reloading instruction is used to resubscribe data of a specific node in a tree structure.

4. The method of claim 3, wherein, The legality of the corresponding instruction is determined, including: checking the node corresponding to the corresponding instruction; when the checking succeeds, determining that the corresponding instruction is legal; when the checking fails, determining that the corresponding instruction is not legal.

5. The method of claim 4, wherein, The method further includes: determining whether the received data contains first information, and determining that the type of the corresponding instruction contains a first type when it is determined that the received data contains the first information; the first information indicates the start of full data under a subscription node.

6. The method of claim 4, wherein, The method further includes: determining whether the received data contains first information, and determining that the type of the corresponding instruction contains a second type when it is determined that the received data does not contain the first information; the first information indicates the end of full data under a subscription node.

7. The method of claim 3, wherein: receiving second information, the second information indicating the end of full data under a subscription node; initiating a subscription reloading instruction to the sending end device based on all node-related information cached.

8. A data processing apparatus, characterized by, provided on a sending end device, including: an acquisition unit configured to acquire at least one instruction, the instruction being used to indicate sending data for synchronization to a receiving end device; the at least one instruction contains a first type of instruction and / or a second type of instruction, the first type of instruction being used to subscribe to data of a specific node in a tree structure, and the second type of instruction indicating data in which a synchronization increment has changed; a first processing unit configured to place the acquired at least one instruction in a first queue; a second processing unit configured to read one instruction from the first queue in a first-in-first-out order, acquire data corresponding to the corresponding instruction in response to the read instruction, and place the corresponding instruction and the corresponding data in a second queue; wherein the data corresponding to one instruction is placed in the second queue before the next instruction is read from the first queue; a synchronization unit configured to synchronize the corresponding instruction and the corresponding data in the second queue to the receiving end device in the first-in-first-out order; the second processing unit is configured to determine the type of the instruction read from the first queue; when the type of the read instruction contains the first type, acquire full data under a subscription node by using tree structure data, and place first information, second information, and full data under the subscription node corresponding to the read instruction in the second queue, the first information indicating the start of full data under the subscription node, and the second information indicating the end of full data under the subscription node; when the type of the read instruction contains the second type, acquire data corresponding to the read instruction from the first queue.

9. A data processing apparatus, characterized by, provided on a receiving end device, including: The receiving unit is configured to receive at least one instruction and corresponding data sent by the sending terminal device in sequence, wherein the instruction is used to instruct the receiving terminal device to send data used for synchronization, and the at least one instruction comprises a first type of instruction and / or a second type of instruction, the first type of instruction is used to subscribe to data of a specific node in a tree structure, and the second type of instruction indicates data in which a synchronization increment changes; The third processing unit is configured to determine the legality of each instruction in the at least one instruction, and in a case where the corresponding instruction is legal, store the corresponding data in a tree structure in response to the corresponding instruction; The fourth processing unit is configured to cache the corresponding instruction and the corresponding data in a case where the corresponding instruction is not legal; The fourth processing unit is further configured to, in a case where the type of the corresponding instruction comprises the first type, determine and cache all node related information of the corresponding instruction corresponding to a check failure in a case where the corresponding instruction is not legal, initiate a subscription reload instruction to the sending terminal device based on the cached all node related information, and the subscription reload instruction is used to resubscribe to data of a specific node in a tree structure, or in a case where the type of the corresponding instruction comprises the second type, determine first node related information of the corresponding instruction corresponding to a check failure in a case where the corresponding instruction is not legal, and initiate the subscription reload instruction to the sending terminal device based on the first node related information, and the subscription reload instruction is used to resubscribe to data of a specific node in a tree structure.

10. A transmitting device, comprising: Comprise: A first processor and a first memory for storing a computer program capable of running on the processor, Wherein, the first processor is used to run the computer program, and the steps of the method of claim 1 or 2 are executed.

11. A receiving end device, characterized by Comprise: A second processor and a second memory for storing a computer program capable of running on the processor, Wherein, the second processor is used to run the computer program, and the steps of the method of any one of claims 3 to 7 are executed.

12. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the steps of the method of claim 1 or 2, or realize the steps of the method of any one of claims 3 to 7.

Images