Data processing method, device and system and storage medium

Abstract

The invention relates to a data processing method, device and system and a storage medium. The method comprises the steps of obtaining a target data type of target data to be published; according to the target data type, sending a query request message to a storage manager, the query request message being used for requesting the storage manager to query a node address of a data subscription node corresponding to the target data; receiving a query response message sent by the storage manager according to the query request message, wherein the query response message comprises a node address of the data subscription node; and sending the target data to the data subscription node according to the node address.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a data processing method, device, system and storage medium. Background Art

[0002] Typically, network applications rely on push notifications, also known as "subscriptions" or "observations," where producers (publishers) send messages to topics, which are distributed to all consumers (subscribers) who have expressed interest in those topics. This design pattern decouples data producers from consumers, allowing for scalable and flexible data distribution.

[0003] In the relevant data subscription, the data is usually published by the producer (publisher) to the message middleware, such as open source Kafka, Bigpipe, etc., and the downstream devices then subscribe to the data from the message middleware. However, the configuration of the message middleware consumes a lot of machine resources; moreover, the message middleware will cause data transmission delays. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a data processing method, device, system and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a data processing method is provided, the method comprising:

[0006] Get the target data type of the target data to be published;

[0007] Sending a query request message to a storage manager according to the target data type, wherein the query request message is used to request the storage manager to query a node address of a data subscription node corresponding to the target data;

[0008] receiving a query response message sent by the storage manager according to the query request message, wherein the query response message includes a node address of the data subscription node;

[0009] Optionally, sending the target data to the data subscription node according to the node address includes:

[0010] Establishing a communication connection with the data subscription node according to the node address;

[0011] Based on the communication connection, the target data is sent to the data subscription node.

[0012] Optionally, sending a query request message to the storage manager according to the target data type includes:

[0013] Add the target data type to a queue to be queried;

[0014] According to the queue to be queried, query request messages are sent to the storage manager in sequence.

[0015] Optionally, the method further comprises:

[0016] When the target data is sent successfully, the target data type stored in the queue to be queried is deleted.

[0017] Optionally, the method further comprises:

[0018] In the case where the target data fails to be sent, sending an indication message to the storage manager, wherein the indication message is used to indicate that the target data has not been published; or

[0019] In the event that the target data fails to be sent, the steps of obtaining the node address of the data subscription node through the storage manager according to the target data type and sending the target data to the data subscription node according to the node address are re-executed.

[0020] According to a second aspect of an embodiment of the present disclosure, a data processing method is provided, the method comprising:

[0021] Receiving a query request message sent by a data publishing node, wherein the query request message is used to request to query a node address of the data subscription node;

[0022] Determine the node address of the data subscription node according to the query request message;

[0023] A query response message is sent to the data publishing node, wherein the query response message includes the node address of the data subscription node, so that the data publishing node sends the target data to the data subscription node according to the node address.

[0024] Optionally, the method further comprises:

[0025] receiving a subscription request message from the data subscription node, wherein the subscription request message includes the target data type subscribed by the data subscription node and the node address of the data subscription node;

[0026] According to the subscription request message, the target data type and the corresponding node address of the data subscription node are stored in the storage manager.

[0027] According to a third aspect of an embodiment of the present disclosure, a data processing method is provided, the method comprising:

[0028] Receive target data sent by a data publishing node according to a node address of a data subscription node; the node address is included in a query response message, the query response message is sent by the storage manager to the data publishing node according to a query request message, and the query request message is sent by the data publishing node to the storage manager according to a target data type of the target data to be published.

[0029] Optionally, the method further comprises:

[0030] Send a subscription request message to the storage manager, the subscription request message including the target data type subscribed by the data subscription node and the node address of the data subscription node

[0031] According to a fourth aspect of an embodiment of the present disclosure, a data processing device is provided, the device comprising:

[0032] A first acquisition module is configured to acquire a target data type of target data to be published;

[0033] A first sending module is configured to send a query request message to a storage manager according to the target data type, wherein the query request message is used to request the storage manager to query a node address of a data subscription node corresponding to the target data;

[0034] A first receiving module is configured to receive a query response message sent by the storage manager according to the query request message, wherein the query response message includes a node address of the data subscription node;

[0035] The second sending module is configured to send the target data to the data subscription node according to the node address.

[0036] Optionally, the second sending module includes:

[0037] A communication submodule, configured to establish a communication connection with the data subscription node according to the node address;

[0038] The first sending submodule is configured to send the target data to the data subscription node based on the communication connection.

[0039] Optionally, the first sending module includes:

[0040] An adding submodule, configured to add the target data type to a queue to be queried;

[0041] The second sending submodule is configured to send query request messages to the storage manager in sequence according to the queue to be queried.

[0042] Optionally, the first sending module further includes:

[0043] The deleting submodule is configured to delete the target data type stored in the queue to be queried when the target data is sent successfully.

[0044] Optionally, the device further comprises:

[0045] A third sending module is configured to send an indication message to the storage manager when the target data fails to be sent, wherein the indication message is used to indicate that the target data has not been published; or

[0046] The execution module is configured to re-execute the steps of obtaining the node address of the data subscription node through the storage manager according to the target data type, and sending the target data to the data subscription node according to the node address when the target data fails to be sent.

[0047] According to a fifth aspect of an embodiment of the present disclosure, a data processing device is provided, the device comprising:

[0048] A second receiving module is configured to receive a query request message sent by a data publishing node, wherein the query request message is used to request to query a node address of the data subscription node;

[0049] A first determining module is configured to determine a node address of the data subscription node according to the query request message,

[0050] The third sending module is configured to send a query response message to the data publishing node, wherein the query response message includes the node address of the data subscription node, so that the data publishing node sends the target data to the data subscription node according to the node address.

[0051] Optionally, the device further comprises:

[0052] A receiving module, configured to receive a subscription request message from the data subscription node, wherein the subscription request message includes a target data type subscribed by the data subscription node and a node address of the data subscription node;

[0053] The storage module is configured to store the target data type and the corresponding node address of the data subscription node in the storage manager according to the subscription request message.

[0054] According to a sixth aspect of an embodiment of the present disclosure, a data processing device is provided, wherein the device is configured to receive target data sent by a data publishing node according to a node address of a data subscription node; the node address is included in a query response message, the query response message is sent by the storage manager to the data publishing node according to a query request message, and the query request message is sent by the data publishing node to the storage manager according to a target data type of the target data to be published.

[0055] According to a seventh aspect of an embodiment of the present disclosure, a data processing device is provided, the device comprising:

[0056] processor;

[0057] a memory for storing processor-executable instructions;

[0058] Wherein, the processor is configured to implement the steps of the data processing method provided in any one of the first aspect, the second aspect or the third aspect of the present disclosure when executing.

[0059] According to an eighth aspect of an embodiment of the present disclosure, there is provided a data processing system, the data processing system comprising a data publishing node, a data subscription node, and a storage manager;

[0060] The data publishing node is used to implement the steps of the data processing method provided in the first aspect of the present disclosure when executed;

[0061] The data subscription node is used to implement the steps of the data processing method provided in the third aspect of the present disclosure when executed;

[0062] The storage manager is used to implement the steps of the data processing method provided in the second aspect of the present disclosure when executed.

[0063] According to the ninth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the data processing method provided in any one of the first aspect, the second aspect or the third aspect of the present disclosure are implemented.

[0064] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0065] By obtaining the target data type of the target data to be published; according to the target data type, sending a query request message to the storage manager, the query request message is used to request the storage manager to query the node address of the data subscription node corresponding to the target data; receiving the query response message sent by the storage manager according to the query request message, the query response message includes the node address of the data subscription node; according to the node address, sending the target data to the data subscription node. In this way, the target data can be sent to the data subscription node through the node address, which can reduce the consumption of machine resources without adding additional message middleware and other devices in addition to the subscription node device and the publishing node device; and the publishing node device can obtain the node address of the data subscription node through the storage manager, so that the storage manager can act as an intermediate medium between the subscription node device and the publishing node device to perform information exchange between the subscription node device and the publishing node device, complete the publishing and subscription of data, and do not need to transfer data of third-party devices such as message middleware, thereby reducing the data transmission delay and improving the data transmission efficiency.

[0066] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0068] Figure 1 The invention is a flowchart of a data processing method according to an exemplary embodiment.

[0069] Figure 2 The figure is a flowchart of another data processing method according to an exemplary embodiment.

[0070] Figure 3 The figure is a flowchart of another data processing method according to an exemplary embodiment.

[0071] Figure 4 The figure is a flowchart of another data processing method according to an exemplary embodiment.

[0072] Figure 5 It is a block diagram of a data processing device according to an exemplary embodiment.

[0073] Figure 6 is based on Figure 5 The illustrated embodiment shows a block diagram of a second sending module.

[0074] Figure 7 is based on Figure 5 The illustrated embodiment shows a block diagram of a first sending module.

[0075] Figure 8 is based on Figure 7 The illustrated embodiment shows a block diagram of a first sending module.

[0076] Fig. 9 is based on Figure 5 The illustrated embodiment shows a block diagram of a data processing device.

[0077] Fig.10 It is a block diagram of a data processing device according to an exemplary embodiment.

[0078] Fig.11 is based on Fig.10 The illustrated embodiment shows a block diagram of another data processing device.

[0079] Fig.12 is a block diagram of a data processing system according to an exemplary embodiment.

[0080] Fig.13 The invention is a block diagram showing a device of a data processing method for a data publishing node according to an exemplary embodiment.

[0081] Fig.14 The invention is a block diagram showing a device for a data processing method of a storage manager according to an exemplary embodiment. DETAILED DESCRIPTION

[0082] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0083] Before introducing the specific implementation methods of the present disclosure in detail, the application scenarios of the present disclosure are first described. The present disclosure can be applied to scenarios where data publishing devices publish data. Generally, Internet of Things (IoT) and industrial IoT applications rely on push notifications, also known as "subscriptions" or "observations", in which an entity ("subscriber" or "consumer") records its attention to certain data (i.e., data of interest) on another entity ("publisher" or "producer"), and the latter entity provides the data of interest and can notify the former entity of any relevant changes (such as events or new states).

[0084] However, in the relevant data subscription, data is usually published by "publishers" or "producers" to message middleware, such as open source Kafka, Bigpipe, etc., and downstream devices ("subscribers" or "consumers") then subscribe to the data from the message middleware. However, the configuration of the message middleware consumes a lot of machine resources; moreover, the message middleware will cause data transmission delays.

[0085] In order to overcome the technical problems existing in the above related technologies, the present disclosure provides a data processing method, device, system and storage medium. By obtaining the target data type of the target data to be published; according to the target data type, by obtaining the target data type of the target data to be published; according to the target data type, a query request message is sent to the storage manager, the query request message is used to request the storage manager to query the node address of the data subscription node corresponding to the target data; receive the query response message sent by the storage manager according to the query request message, the query response message includes the node address of the data subscription node; according to the node address, the target data is sent to the data subscription node. In this way, the target data can be sent to the data subscription node through the node address, and there is no need to add additional message middleware and other devices outside the subscription node device and the publishing node device, which can reduce machine resource consumption; and the publishing node device can obtain the node address of the data subscription node through the storage manager, so that the storage manager can be used as an intermediate medium between the subscription node device and the publishing node device to perform information interaction between the subscription node device and the publishing node device, complete the publishing and subscription of data, and do not need to transfer data of third-party devices such as message middleware, thereby reducing data transmission delay and improving data transmission efficiency.

[0086] The present disclosure is described below in conjunction with specific embodiments.

[0087] Figure 1 is a flow chart of a data processing method according to an exemplary embodiment. Figure 1 As shown, the method is applied to a data publishing node of a data processing system, the data processing system includes the data publishing node, a data subscription node, and a storage manager, and the method includes the following steps.

[0088] In step S101 , the target data type of the target data to be published is obtained.

[0089] In this step, the data publishing node may generate target data for publishing, such as but not limited to logs, business data, heartbeat data, etc., and then publish the generated target data.

[0090] For example, the target data may include a target data type. For example, the target data may be expressed as {[user]; [A, B, C, D...]}, where [user] is used to characterize the target data type, and [A, B, C, D...] is used to characterize the target data content, including user A, user B, user C, user D, and so on.

[0091] Among them, in one possible implementation method, the data publishing node can publish the acquired target data to be published in real time or periodically to publish the target data; in another possible implementation method, when it is determined that the storage manager stores the target data type corresponding to the target data subscription, the target data to be published can be acquired to publish the target data.

[0092] For example, when the storage manager stores a target data type corresponding to the target data subscribed to, a data request message can be broadcast. The data request message can be used to request a data publishing node including the target data type to publish the target data. When the data publishing node receives the data request message, it can obtain the target data type of the target data to be published in order to publish the target data.

[0093] In step S102, a query request message is sent to a storage manager according to the target data type.

[0094] The query request message is used to request the storage manager to query the node address of the data subscription node corresponding to the target data. The data subscription node may be a node that subscribes to the target data type, and the storage manager stores the node addresses of nodes corresponding to different data types.

[0095] In this step, when the data publishing node obtains the target data type of the target data to be published, it can obtain the node address of the data subscription node stored on the storage manager by sending a query request message to the storage manager, where the data subscription node is a node that subscribes to the target data type.

[0096] In a possible implementation, the storage manager can be set as a distributed shared storage service manager, which can include multiple storage service sub-managers, each of which can be connected to a different node, and when data changes occur on any of the storage service sub-managers, the remaining storage service sub-managers will synchronously make the same data changes. In other words, the data stored on each storage service sub-manager is the same.

[0097] In this step, the data publishing node may first send a query request message to the storage service sub-manager connected to itself, so as to request the storage service sub-manager to query the node address of the data subscription node.

[0098] In some embodiments, the data publishing node may first add the target data type to a queue to be queried, and then may send query request messages to the storage manager in sequence according to the queue to be queried.

[0099] By way of example, the data publishing node may also include a node manager, which is used to receive the target data type obtained by the data publishing node, and add the target data type as a to-be-queried message to the to-be-queried queue of the node manager, and then send query request messages to the storage manager in turn.

[0100] Optionally, when the target data is sent successfully, the target data type stored in the queue to be queried is deleted.

[0101] In step S103, a query response message sent by the storage manager according to the query request message is received.

[0102] The query response message includes the node address of the data subscription node.

[0103] Optionally, the data publishing node may receive a query response message sent by the storage manager, and determine the node address of the data subscription node according to the query response message.

[0104] In this step, when the storage manager receives the query request message sent by the data publishing node, it can determine the node address of the data subscription node corresponding to the target data type from the node addresses of the nodes corresponding to different data types stored in itself according to the target data type carried in the query request message, wherein the data subscription node is a node that subscribes to the target data type.

[0105] By adopting the above technical solution, a query response message can be sent to the storage manager through the data publishing node to obtain the node address of the data subscription node. In this way, the data subscription node can be directly connected based on the node address of the data subscription node, which can reduce data transmission delay and improve data transmission efficiency.

[0106] In step S104, the target data is sent to the data subscription node according to the node address.

[0107] In this step, when the data publishing node obtains the node address of the data subscription node, it can first establish a communication connection with the data subscription node according to the node address, and then send the target data to the data subscription node based on the communication connection.

[0108] By adopting the above technical solution, the target data type of the target data to be published can be obtained; according to the target data type, a query request message is sent to the storage manager, and the query request message is used to request the storage manager to query the node address of the data subscription node corresponding to the target data; the query response message sent by the storage manager according to the query request message is received, and the query response message includes the node address of the data subscription node; according to the node address, the target data is sent to the data subscription node. In this way, the target data can be sent to the data subscription node through the node address, and there is no need to add additional message middleware and other devices in addition to the subscription node device and the publishing node device, which can reduce the consumption of machine resources; and the publishing node device can obtain the node address of the data subscription node through the storage manager, so that the storage manager can act as an intermediate medium between the subscription node device and the publishing node device to perform information interaction between the subscription node device and the publishing node device, complete the publishing and subscription of data, and do not need to transfer data of third-party devices such as message middleware, thereby reducing the data transmission delay and improving the data transmission efficiency.

[0109] In some embodiments, Figure 2 As shown, the method may further include the following steps.

[0110] In step S104, when the target data fails to be sent, an indication message is sent to the storage manager, where the indication message is used to indicate that the target data has not been published.

[0111] In this step, if the target data fails to be sent during the sending process, for example, the data subscription node is temporarily unreachable, or the data publishing node fails to obtain the node address of the node that subscribes to the target data type from the storage manager, in this case, the data publishing node can send an indication message to the storage manager, or it can treat the target data type as a message to be queried and put the message to be queried into a "failed message" queue for subsequent processing.

[0112] Alternatively, in step S105, when the target data fails to be sent, the steps of obtaining the node address of the data subscription node through the storage manager according to the target data type and sending the target data to the data subscription node according to the node address are re-executed.

[0113] By adopting the above technical solution, when the target data fails to be sent, a data sending failure reminder can be issued, or the data can be re-published.

[0114] Figure 3 is a flow chart of another data processing method according to an exemplary embodiment. Figure 3 As shown, the method is applied to a storage manager of a data processing system, the data processing system includes a data publishing node, a data subscription node, and the storage manager, and the method includes the following steps.

[0115] In step S201, a query request message sent by a data publishing node is received, where the query request message is used to request to query the node address of the data subscription node.

[0116] In a possible implementation, the storage manager can be set as a distributed shared storage service manager, which can include multiple storage service sub-managers, each of which can be connected to a different node, and when data changes occur on any of the storage service sub-managers, the remaining storage service sub-managers will synchronously make the same data changes. In other words, the data stored on each storage service sub-manager is the same.

[0117] In this step, each storage service sub-manager in the storage manager may monitor the connected data publishing node periodically or in real time, and receive the query request message sent by the data publishing node when the data publishing node sends a query request message.

[0118] In step S202, the node address of the data subscription node is determined according to the query request message.

[0119] In this step, when the storage manager receives the query request message, it can determine the node address of the data subscription node corresponding to the target data type from the node addresses of the nodes corresponding to different data types stored in itself, based on the target data type carried in the query request message, wherein the data subscription node is a node that subscribes to the target data type.

[0120] In step S203, a query response message is sent to the data publishing node.

[0121] The query response message includes the node address of the data subscription node, so that the data publishing node sends the target data to the data subscription node according to the node address.

[0122] By adopting the above technical solution, the target data type of the target data to be published can be obtained; according to the target data type, a query request message is sent to the storage manager, and the query request message is used to request the storage manager to query the node address of the data subscription node corresponding to the target data; the query response message sent by the storage manager according to the query request message is received, and the query response message includes the node address of the data subscription node; according to the node address, the target data is sent to the data subscription node. In this way, the target data can be sent to the data subscription node through the node address, and there is no need to add additional message middleware and other devices in addition to the subscription node device and the publishing node device, which can reduce machine resource consumption; and the publishing node device can obtain the node address of the data subscription node through the storage manager, so that the storage manager can act as an intermediate medium between the subscription node device and the publishing node device to perform information exchange between the subscription node device and the publishing node device, complete the publishing and subscription of data, and do not need to transfer data of third-party devices such as message middleware, thereby reducing data transmission delay and improving data transmission efficiency.

[0123] In some embodiments, the storage manager may first obtain and store the target data type and the corresponding node address of the data subscription node before determining the node address of the data subscription node based on the query request message, wherein obtaining and storing the target data type and the corresponding node address of the data subscription node may be achieved in the following manner.

[0124] First, a subscription request message of the data subscription node is received; then, according to the subscription request message, the target data type and the corresponding node address of the data subscription node can be stored in the storage manager.

[0125] The subscription request message includes the target data type subscribed by the data subscription node and the node address of the data subscription node.

[0126] In one possible implementation, the storage manager may monitor periodically or receive subscription request messages in real time. The subscription request message may be a request message sent by the data subscription node to the storage manager, indicating that the data subscription node subscribes to the target data type. When the storage manager receives the subscription request message from the data subscription node, the target data type and the corresponding node address of the data subscription node carried in the subscription request message may be stored in the storage manager. Then, the storage manager may reply a response message to the data subscription node to indicate that the target data type and the corresponding node address of the data subscription node have been successfully stored.

[0127] In some embodiments, the data processing method can also be applied to a data subscription node of a data processing system, which includes a data publishing node, the data subscription node, and a storage manager. When the data processing method is applied to the data subscription node of the data processing system, the data subscription node can receive target data sent by the data publishing node according to the node address of the data subscription node.

[0128] The node address is included in a query response message, which is sent by the storage manager to the data publishing node according to a query request message, and the query request message is sent by the data publishing node to the storage manager according to a target data type of the target data to be published.

[0129] Figure 4 is a flow chart of another data processing method according to an exemplary embodiment. Figure 4 As shown, the method is applied to a data processing system, which includes a data publishing node, a data subscription node, and a storage manager. The method includes the following steps.

[0130] In step S301, the data subscription node obtains the target data type.

[0131] The target data type is the data type subscribed by the data subscription node, and the data subscription node is the node that subscribes to the target data type.

[0132] In step S302, the data subscription node sends a subscription request message to the storage manager.

[0133] The subscription request message includes the target data type subscribed by the data subscription node and the node address of the data subscription node. The storage manager stores the node addresses of nodes corresponding to different data types.

[0134] In step S303, the storage manager receives a subscription request message from the data subscription node.

[0135] In step S304, according to the subscription request message, the target data type and the corresponding node address of the data subscription node are stored in the storage manager.

[0136] In step S305 , the data publishing node obtains the target data type of the target data to be published.

[0137] In step S306, the data publishing node adds the target data type to the queue to be queried.

[0138] In step S307, the data publishing node sends query request messages to the storage manager in sequence according to the queue to be queried.

[0139] The query request message is used to request the storage manager to query the node address of the data subscription node.

[0140] In step S308, the storage manager receives the query request message sent by the data publishing node.

[0141] The query request message is used to request to query the node address of the data subscription node.

[0142] In step S309, the storage manager determines the node address of the data subscription node according to the query request message.

[0143] In step S310, the storage manager sends a query response message to the data publishing node.

[0144] The query response message includes the node address of the data subscription node.

[0145] In step S311, the data publishing node receives the query response message sent by the storage manager, and determines the node address of the data subscription node according to the query response message.

[0146] In step S312, the data publishing node establishes a communication connection with the data subscription node according to the node address.

[0147] In step S313, based on the communication connection, the data publishing node sends the target data to the data subscription node.

[0148] In step S314, the data subscription node is used to receive the target data sent by the data publishing node.

[0149] In some embodiments, when the target data is sent successfully, the target data type stored in the queue to be queried is deleted; when the target data is sent unsuccessfully, an indication message is sent to the storage manager, the indication message being used to indicate that the target data has not been published; or, steps S305 to S314 are re-executed.

[0150] By adopting the above technical scheme, the target data type of the target data to be published is obtained through the data publishing node; according to the target data type, the node address of the data subscription node is obtained through the storage manager, the data subscription node is a node that subscribes to the target data type, and the storage manager stores the node addresses of nodes corresponding to different data types; according to the node address, the target data is sent to the data subscription node, and a query request message sent by the data publishing node is received through the storage manager, the query request message is used to request to query the node address of the data subscription node; according to the query request message, the node address of the data subscription node is determined; and a query response message is sent to the data publishing node, the query response message includes the node address of the data subscription node, so that the data publishing node sends the target data to the data subscription node according to the node address. In this way, the target data can be sent to the data subscription node through the node address, and there is no need to add additional message middleware and other devices in addition to the subscription node device and the publishing node device, which can reduce machine resource consumption; and the publishing node device can obtain the node address of the data subscription node through the storage manager, so that the storage manager can act as an intermediate medium between the subscription node device and the publishing node device to exchange information between the subscription node device and the publishing node device, complete data publishing and subscription, and do not need third-party devices such as message middleware to transfer data, thereby reducing data transmission delay and improving data transmission efficiency.

[0151] The implementation of the present disclosure is described below with reference to a specific example.

[0152] The method disclosed in the present application can be applied to a data processing system, which includes a data publishing node, a data subscription node, and a storage manager. For example, device A can be used as the data publishing node, and device B can be used as the data subscription node.

[0153] First, the application framework on device A generates the target data to be published, such as a new message with the content "hello". This message is marked as topic "user" and packaged with relevant meta-information (for example, topic, timestamp); then the application framework of device A sends this packaged message to the publish-subscribe manager on device A, which checks the integrity and format of the message. After confirming that it is correct, it adds the message to the local queue to be published of the publish-subscribe manager on device A.

[0154] The publish-subscribe manager of device A reads the messages in the queue to be published periodically or based on a preset trigger mechanism, and when the "user" topic message is identified, the publish-subscribe manager of device A can send a query request to the storage manager to inquire about all data subscription nodes that have subscribed to the "user" topic. The storage manager can then return a list of nodes that have subscribed to the "user" topic, including the address of device B. The publish-subscribe manager of device A then uses a predefined communication protocol to send a "hello" message to device B. After the message is sent successfully, the message is deleted from the queue to be published of device A.

[0155] The Publish and Subscribe Manager on device B listens to and successfully receives the "hello" message from device A. The Publish and Subscribe Manager on device B then verifies the integrity and format of the message. After the verification, the Publish and Subscribe Manager on device B adds the "hello" message to the received message queue on device B. The Publish and Subscribe Manager on device B can then notify its upper-layer application framework that a new "user" topic message is available for processing, and then the "hello" message is read and processed by the application framework on device B. After processing, the message is deleted from the received message queue of device B.

[0156] It should be noted that before the Publish-Subscribe Manager on device B listens to and successfully receives the "hello" message from device A, the application framework (or some other upper-layer application) on device B decides that it needs to subscribe to the "user" topic, and can send a subscription request to the Publish-Subscribe Manager on device B. This request will clearly indicate that the topic to be subscribed is "user". Then the Publish-Subscribe Manager of device B will then send a request to the storage manager to add device B to the subscription node list of the "user" topic. The storage manager can update its stored subscriber list based on the node address of device B, add the address of device B to the subscriber list of the "user" topic, and the storage manager can return a confirmation message to the Publish-Subscribe Manager of device B, indicating that the subscription list has been successfully updated. The Publish-Subscribe Manager of device B then passes the confirmation message to the upper-layer application or component to inform it that it has successfully subscribed to the "user" topic. Once the subscription is successful, the Publish-Subscribe Manager of device B will start listening for messages related to the "user" topic. When a publisher publishes a message on this topic, the manager of device B will receive it and process it further.

[0157] Figure 5 is a block diagram of a data processing device according to an exemplary embodiment. Figure 5The device includes a first acquisition module 501, a first sending module 502, a first receiving module 503 and a second sending module 504.

[0158] A first acquisition module 501 is configured to acquire a target data type of target data to be published;

[0159] A first sending module 502 is configured to send a query request message to a storage manager according to the target data type, wherein the query request message is used to request the storage manager to query a node address of a data subscription node corresponding to the target data;

[0160] A first receiving module 503 is configured to receive a query response message sent by the storage manager according to the query request message, wherein the query response message includes a node address of the data subscription node;

[0161] The second sending module 504 is configured to send the target data to the data subscription node according to the node address.

[0162] Figure 6 is based on Figure 5 The embodiment shown is a block diagram of a second sending module. Figure 6 , the second sending module 504 includes:

[0163] The communication submodule 5041 is configured to establish a communication connection with the data subscription node according to the node address;

[0164] The first sending submodule 5042 is configured to send the target data to the data subscription node based on the communication connection.

[0165] Figure 7 is based on Figure 5 The embodiment shown is a block diagram of a first sending module. Figure 7 , the first sending module 502 includes:

[0166] An adding submodule 5021 is configured to add the target data type to a queue to be queried;

[0167] The second sending submodule 5022 is configured to send query request messages to the storage manager in sequence according to the queue to be queried.

[0168] Figure 8 is based on Figure 7 The embodiment shown is a block diagram of a first sending module. Figure 8 , the first sending module 502 further includes:

[0169] The deleting submodule 5023 is configured to delete the target data type stored in the queue to be queried when the target data is sent successfully.

[0170] Fig. 9 is based on Figure 5 The embodiment shown is a block diagram of a data processing device. Fig. 9 , the device further comprises:

[0171] The third sending module 505 is configured to send an indication message to the storage manager when the target data fails to be sent, where the indication message is used to indicate that the target data has not been published; or

[0172] The execution module 506 is configured to re-execute the steps of obtaining the node address of the data subscription node through the storage manager according to the target data type, and sending the target data to the data subscription node according to the node address when the target data fails to be sent.

[0173] Fig.10 is a block diagram of a data processing device according to an exemplary embodiment. Fig.10 , the device comprises:

[0174] The second receiving module 1001 is configured to receive a query request message sent by a data publishing node, where the query request message is used to request a node address of the data subscription node;

[0175] The first determining module 1002 is configured to determine the node address of the data subscription node according to the query request message,

[0176] The third sending module 1003 is configured to send a query response message to the data publishing node, where the query response message includes the node address of the data subscription node, so that the data publishing node sends the target data to the data subscription node according to the node address.

[0177] Fig.11 is based on Fig.10 Another data processing device block diagram of the embodiment shown. Fig.11 , the device further comprises:

[0178] The receiving module 1004 is configured to receive a subscription request message from the data subscription node, where the subscription request message includes the target data type subscribed by the data subscription node and the node address of the data subscription node;

[0179] The storage module 1005 is configured to store the target data type and the corresponding node address of the data subscription node to the storage manager according to the subscription request message.

[0180] Optionally, the data processing device can also be applied to a data subscription node of a data processing system, which includes a data publishing node, the data subscription node, and a storage manager. When the data processing device is applied to the data subscription node of the data processing system, the data subscription node can receive target data sent by the data publishing node according to the node address of the data subscription node.

[0181] The node address is included in a query response message, which is sent by the storage manager to the data publishing node according to a query request message, and the query request message is sent by the data publishing node to the storage manager according to a target data type of the target data to be published.

[0182] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the steps of the data processing method applied to a data processing system in the present disclosure are implemented.

[0183] Fig.12 is a block diagram of a data processing system according to an exemplary embodiment. Fig.12 , the data processing system includes a data publishing node 1201, a data subscription node 1202, and a storage manager 1203;

[0184] The data publishing node 1201 is used to implement the steps of the data processing method provided in the first aspect of the present disclosure when executed;

[0185] The data subscription node 1202 is used to implement the steps of the data processing method provided in the third aspect of the present disclosure when executed;

[0186] The storage manager 1203 is used to implement the steps of the data processing method provided in the second aspect of the present disclosure when executed.

[0187] Fig.13 1 is a block diagram of a device 1300 for a data publishing node data processing method according to an exemplary embodiment. For example, the device 1300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0188] Reference Fig.13 , the device 1300 may include one or more of the following components: a processing component 1302 , a memory 1304 , a power component 1306 , a multimedia component 1308 , an audio component 1310 , an input / output interface 1312 , a sensor component 1314 , and a communication component 1316 .

[0189] The processing component 1302 generally controls the overall operation of the device 1300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1302 may include one or more processors 1320 to execute instructions to complete all or part of the steps of the data processing method applied to the data publishing node of the data processing system. In addition, the processing component 1302 may include one or more modules to facilitate the interaction between the processing component 1302 and other components. For example, the processing component 1302 may include a multimedia module to facilitate the interaction between the multimedia component 1308 and the processing component 1302.

[0190] The memory 1304 is configured to store various types of data to support the operation of the device 1300. Examples of such data include instructions for any application or method operating on the device 1300, contact data, phone book data, messages, pictures, videos, etc. The memory 1304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0191] The power supply component 1306 provides power to the various components of the device 1300. The power supply component 1306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1300.

[0192] The multimedia component 1308 includes a screen that provides an output interface between the device 1300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1308 includes a front camera and / or a rear camera. When the device 1300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0193] The audio component 1310 is configured to output and / or input audio signals. For example, the audio component 1310 includes a microphone (MIC), and when the device 1300 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1304 or sent via the communication component 1316. In some embodiments, the audio component 1310 also includes a speaker for outputting audio signals.

[0194] The input / output interface 1312 provides an interface between the processing component 1302 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0195] The sensor assembly 1314 includes one or more sensors for providing various aspects of the status assessment of the device 1300. For example, the sensor assembly 1314 can detect the open / closed state of the device 1300, the relative positioning of components, such as the display and keypad of the device 1300, the sensor assembly 1314 can also detect the position change of the device 1300 or a component of the device 1300, the presence or absence of user contact with the device 1300, the orientation or acceleration / deceleration of the device 1300, and the temperature change of the device 1300. The sensor assembly 1314 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1314 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1314 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0196] The communication component 1316 is configured to facilitate wired or wireless communication between the device 1300 and other devices. The device 1300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0197] In an exemplary embodiment, the apparatus 1300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute a data processing method applied to a data publishing node of a data processing system.

[0198] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1304 including instructions, which can be executed by a processor 1320 of the device 1300 to complete a data processing method applied to a data publishing node of a data processing system. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0199] Fig.14 1 is a block diagram of a device 1400 for a data processing method for a storage manager according to an exemplary embodiment. For example, the device 1400 may be provided as a server. Fig.14 , the apparatus 1400 includes a processing component 1422, which further includes one or more processors, and a memory resource represented by a memory 1432 for storing instructions, such as an application, that can be executed by the processing component 1422. The application stored in the memory 1432 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1422 is configured to execute instructions to perform the data processing method for a storage manager of a data processing system as described above.

[0200] The device 1400 may also include a power supply component 1426 configured to perform power management of the device 1400, a wired or wireless network interface 1450 configured to connect the device 1400 to a network, and an input / output interface 1458. The device 1400 may operate based on an operating system stored in the memory 1432, such as Windows Server 2003. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0201] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0202] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: The method comprises: Get the target data type of the target data to be published; Sending a query request message to a storage manager according to the target data type, wherein the query request message is used to request the storage manager to query a node address of a data subscription node corresponding to the target data; receiving a query response message sent by the storage manager according to the query request message, wherein the query response message includes a node address of the data subscription node; The target data is sent to the data subscription node according to the node address.

2. The method according to claim 1, characterized in that The sending the target data to the data subscription node according to the node address comprises: Establishing a communication connection with the data subscription node according to the node address; Based on the communication connection, the target data is sent to the data subscription node.

3. The method according to claim 1, characterized in that The sending a query request message to the storage manager according to the target data type includes: Add the target data type to a queue to be queried; According to the queue to be queried, query request messages are sent to the storage manager in sequence.

4. The method according to claim 3, characterized in that: The method further comprises: When the target data is sent successfully, the target data type stored in the queue to be queried is deleted.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: In the case where the target data fails to be sent, sending an indication message to the storage manager, wherein the indication message is used to indicate that the target data has not been published; or In the event that the target data fails to be sent, the steps of obtaining the node address of the data subscription node through the storage manager according to the target data type and sending the target data to the data subscription node according to the node address are re-executed.

6. A data processing method, characterized in that: The method comprises: Receiving a query request message sent by a data publishing node, wherein the query request message is used to request to query a node address of the data subscription node; Determine the node address of the data subscription node according to the query request message; A query response message is sent to the data publishing node, wherein the query response message includes the node address of the data subscription node, so that the data publishing node sends the target data to the data subscription node according to the node address.

7. The method according to claim 6, characterized in that The method further comprises: receiving a subscription request message from the data subscription node, wherein the subscription request message includes the target data type subscribed by the data subscription node and the node address of the data subscription node; According to the subscription request message, the target data type and the corresponding node address of the data subscription node are stored in the storage manager.

8. A data processing method, characterized in that: The method comprises: Receive target data sent by a data publishing node according to a node address of a data subscription node; the node address is included in a query response message, the query response message is sent by the storage manager to the data publishing node according to a query request message, and the query request message is sent by the data publishing node to the storage manager according to a target data type of the target data to be published.

9. The method according to claim 8, characterized in that The method further comprises: A subscription request message is sent to the storage manager, where the subscription request message includes the target data type subscribed by the data subscription node and the node address of the data subscription node.

10. A data processing device, characterized in that: The device comprises: A first acquisition module is configured to acquire a target data type of target data to be published; A first sending module is configured to send a query request message to a storage manager according to the target data type, wherein the query request message is used to request the storage manager to query a node address of a data subscription node corresponding to the target data; A first receiving module is configured to receive a query response message sent by the storage manager according to the query request message, wherein the query response message includes a node address of the data subscription node; The second sending module is configured to send the target data to the data subscription node according to the node address.

11. A data processing device, characterized in that: The device comprises: A second receiving module is configured to receive a query request message sent by a data publishing node, wherein the query request message is used to request to query a node address of the data subscription node; A first determining module is configured to determine a node address of the data subscription node according to the query request message, The third sending module is configured to send a query response message to the data publishing node, wherein the query response message includes the node address of the data subscription node, so that the data publishing node sends the target data to the data subscription node according to the node address.

12. A data processing device, characterized in that: The device comprises: The third receiving module is configured to receive target data sent by the data publishing node according to a query response message; the query response message includes the node address of the data subscription node, and the query response message is the target data type of the target data to be published acquired by the data publishing node, and after sending a query request message to the storage manager, the storage manager receives the node address of the data subscription node determined according to the query request message.

13. A data processing device, characterized in that: The device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to implement the steps of the method described in any one of claims 1-5, claims 6-7 or claims 8-9 when executing.

14. A data processing system, characterized in that: The data processing system includes a data publishing node, a data subscription node, and a storage manager; The data publishing node is used to execute the method according to any one of claims 1 to 5; The data subscription node is used to execute the method according to any one of claims 6 or 7; The storage manager is used to execute the method according to any one of claims 8 or 9.

15. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1-5, claims 6-7 or claims 8-9 are implemented.

Images