Data processing method, device, apparatus, and computer storage medium

Abstract

The application relates to the technical field of Fintech, and discloses a data processing method, which comprises the following steps: determining all rules corresponding to each node according to a theme in all nodes, constructing a directed graph based on new rules in each rule, and detecting whether a loop exists in the directed graph; if no loop exists in the directed graph, receiving all events sent by a client, determining target rules corresponding to each event based on each rule; determining a sequence of each event according to the target rules corresponding to each event, detecting whether a common sequence exists in each sequence; if the common sequence exists, determining all target events corresponding to the common sequence, and merging data of each target event in the common sequence. The application also discloses a data processing device, equipment and a computer storage medium. The application reduces the probability of repetitive event flow to downstream nodes and avoids resource waste.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Fintech, and in particular to a data processing method and device, equipment and a computer storage medium. BACKGROUND

[0002] With the development of computer technology, more and more technologies (big data, distributed, Blockchain, artificial intelligence, etc.) are applied in the financial field, and the traditional financial industry is gradually changing to Fintech. However, due to the security and real-time requirements of the financial industry, higher requirements are also put forward for data processing technology. In each application scenario of the Internet of Things industry, Internet of Things devices are often applied. When users use the Internet of Things devices, a series of instructions (or events) will be set, but there will be a phenomenon that the user sets repetitive messages or content overlapping instructions (i.e. repetitive events) flow to downstream nodes, resulting in a waste of a large number of resources such as machines and bandwidth, and sending repeated information to the client held by the user will also affect the user's use. Therefore, how to reduce the flow of repetitive events to downstream nodes and avoid resource waste has become a problem that needs to be solved at present. SUMMARY

[0003] The main purpose of the present application is to provide a data processing method, device, equipment and computer storage medium, which aims to solve the technical problem of how to reduce the flow of repetitive events to downstream nodes and avoid resource waste.

[0004] To achieve the above purpose, the present application provides a data processing method, which comprises the following steps:

[0005] Determine all rules corresponding to each node according to the theme in all nodes, construct a directed graph based on new rules in each rule, and detect whether there is a loop in the directed graph;

[0006] If there is no loop in the directed graph, receive all events sent by the client, determine the target rule corresponding to each event based on each rule;

[0007] Determine the sequence of each event according to the target rule corresponding to each event, and detect whether there is a common sequence in each sequence;

[0008] If there is a common sequence, determine all target events corresponding to the common sequence, and merge the data of each target event in the common sequence.

[0009] Optionally, the step of detecting whether there is a common sequence in each sequence comprises:

[0010] determining all nodes of a target sequence in each of the sequences, and matching all nodes of the target sequence with nodes of all other sequences except the target sequence in each of the sequences in turn;

[0011] if the first node in all nodes of the target sequence matches the second node in all other sequences, determining whether the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are the same;

[0012] detecting whether there is a common sequence in each of the sequences according to a determination result of the determination of whether the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are the same.

[0013] Optionally, the step of merging each of the target events in the data of the common sequence comprises:

[0014] determining event attributes and event content of each of the target events in the common sequence based on a preset threshold time;

[0015] merging repeated content in each of the event content, and merging repeated event attributes in each of the event attributes.

[0016] Optionally, the step of constructing a directed graph based on a new rule in each of the rules comprises:

[0017] determining a target node corresponding to the new rule in each of the rules based on each of the nodes, and taking a subject in the target node as a target subject;

[0018] determining a node flow direction of the new rule in each of the target nodes according to each of the target subjects, connecting each of the target nodes according to the node flow direction, and taking the connected each of the target nodes as a directed graph.

[0019] Optionally, the step of detecting whether there is a loop in the directed graph comprises:

[0020] determining all target nodes in the directed graph, determining a vertex in each of the target nodes according to a node flow direction of each of the target nodes, and traversing each of the target nodes from the vertex according to each of the node flow directions;

[0021] matching the traversed target node with other nodes in each of the target nodes except the traversed target node, and determining whether there is a loop in the directed graph based on a matching result of the matching.

[0022] Optionally, the step of determining a target rule corresponding to each of the events based on each of the rules comprises:

[0023] sequentially traversing each of the events, obtaining an event attribute of a traversed event, and matching the event attribute with each of the rules;

[0024] If there is a matching rule matching the event attribute in each of the rules, the matching rule is taken as a target rule corresponding to the traversed event.

[0025] Optionally, before the step of determining all rules corresponding to each of the nodes according to the topics in all nodes, the method comprises:

[0026] obtaining a plurality of input topics, and sequentially detecting whether each of the topics meets a preset node requirement;

[0027] If each of the topics meets the preset node requirement, determining a node corresponding to each of the topics, and sequentially assigning each of the topics to the node corresponding to the topic.

[0028] In addition, to achieve the above object, the present application also provides a data processing device, which comprises:

[0029] a construction unit configured to determine all rules corresponding to each of the nodes according to the topics in all nodes, construct a directed graph based on new rules in each of the rules, and detect whether there is a loop in the directed graph;

[0030] a determination unit configured to, if there is no loop in the directed graph, receive all events sent by a client, and determine a target rule corresponding to each of the events based on each of the rules;

[0031] a detection unit configured to determine a sequence of each of the events according to the target rule corresponding to each of the events, and detect whether there is a common sequence in each of the sequences;

[0032] a merging unit configured to, if there is a common sequence, determine all target events corresponding to the common sequence, and merge data of each of the target events in the common sequence.

[0033] In addition, to achieve the above object, the present application also provides a data processing device, which comprises a memory, a processor, and a data processing program stored in the memory and executable on the processor, wherein the data processing program is executed by the processor to implement the steps of the data processing method.

[0034] In addition, to achieve the above object, the present application also provides a computer storage medium, wherein the computer storage medium stores a data processing program, and the data processing program is executed by a processor to implement the steps of the data processing method.

[0035] The application determines all rules corresponding to each node according to the theme in all nodes, constructs a directed graph based on new rules in each rule, and detects whether there is a loop in the directed graph; if there is no loop in the directed graph, all events sent by the receiving client are received, target rules corresponding to each event are determined based on each rule; the sequence of each event is determined according to the target rule corresponding to each event, whether there is a common sequence in each sequence is detected; if there is a common sequence, all target events corresponding to the common sequence are determined, and the data of each target event in the common sequence is merged. By determining all rules according to the theme in all nodes, constructing a directed graph according to new rules in each rule, determining target rules corresponding to all events sent by the client when there is no loop in the directed graph, determining the sequence of all events, and merging the data of all target events corresponding to the common sequence in the common sequence when there is a common sequence in each sequence, the phenomenon of the repetitive message or the instruction flow with overlapping content flowing to the downstream node in the prior art is avoided, the resource waste is reduced, and the probability of the repetitive event flowing to the downstream node is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a data processing device structure schematic diagram of a hardware running environment related to an embodiment scheme of the application.

[0037] Figure 2 is a flowchart of a data processing method first embodiment of the application.

[0038] Figure 3 is a device unit schematic diagram of a data processing device of the application.

[0039] Fig. 4(a) is a flowchart of theme creation in the data processing method of the application.

[0040] Fig. 4(b) is a flowchart of rule analysis and event hitting in the data processing method of the application.

[0041] Figure 5 is a flowchart of event sequence in the data processing method of the application.

[0042] Figure 6 is a flowchart of the data processing method of the application.

[0043] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are merely intended to explain the application and are not intended to limit the application.

[0045] AsFigure 1 As shown in the figure, Figure 1 is a schematic diagram of the data processing device structure of the hardware running environment involved in the embodiment of the application.

[0046] The data processing device of the embodiment of the application can be a PC or a server device, and a Java virtual machine runs on the data processing device.

[0047] As shown in the figure, Figure 1 the data processing device can include a processor 1001, for example, a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 can further include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory) such as a disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001.

[0048] Those skilled in the art can understand that Figure 1 the data processing device structure shown in the figure does not constitute a limitation on the device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0049] As shown in the figure, Figure 1 the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a data processing program.

[0050] In the data processing device shown in the figure, Figure 1 the network interface 1004 is mainly used to connect a background server and communicate data with the background server; the user interface 1003 is mainly used to connect a client (user end) and communicate data with the client; and the processor 1001 can be used to call the data processing program stored in the memory 1005 and perform the operations in the following data processing method.

[0051] Based on the above hardware structure, the embodiment of the data processing method of the application is proposed.

[0052] Referring to Figure 2 , Figure 2 is a flowchart of the first embodiment of the data processing method of the application, and the method includes:

[0053] In step S10, all rules corresponding to each node are determined according to the topics in all nodes, a directed graph is constructed based on new rules in each rule, and whether there is a loop in the directed graph is detected.

[0054] As the current user's operation is easy to cause repetitive instructions or overlapping content data flow to downstream nodes, resulting in waste of resources. Therefore, in this embodiment, a directed graph is constructed according to the relationship of the event information flow, and the directed graph is analyzed in real time, so as to ensure that the directed graph formed according to the constructed rules is acyclic. At the same time, when calculating the instruction or event flow, the common sequence of the directed graph corresponding to the event flow is calculated, and the relationship algebra between the parameters of the instructions or events flowing to the common sequence is calculated, such as the intersection, and then a reasonable path for the instructions or events to flow downstream is planned according to the calculated intersection, so as to reduce the repetitive instructions or repetitive event content flowing to the downstream nodes, and avoid the waste of resources such as machines and bandwidth. For example, as shown in FIG. 4(a), including a client 1, an event agent module, a rule engine module and a blockchain node. Among them, the event agent module is responsible for the publication and subscription of events and the access to the blockchain. The rule engine module is used to provide time sequence flow analysis and event linkage for users. First, the client creates a topic in the event agent module, and checks and processes the created topic. When the topic meets the requirements, the topic is written into the blockchain node of the blockchain, and the node consensus of the blockchain node is performed, and then the successful prompt information is returned to the client. Then set the rule engine to construct the directed graph of the rule, and analyze the directed graph to determine whether there is a loop. If there is no loop, it is determined that the rule engine is set successfully. Moreover, the event agent module will listen to the rules in the rule engine module to determine whether the rule engine is available. When the event agent module determines that the rule engine is available according to the received return data, the event sent by the client 1 is received, and the event content is listened to and obtained. The event and the rule in the rule engine are matched to determine whether they hit. If a rule is hit, the event is flowed according to the rule content of the rule, and the successful prompt information is returned to the client 1. Moreover, the analysis process of the directed graph and the process of the event rule matching can be as shown in FIG. 4(b), including a client, a detection module, a cache module and a calculation and analysis module. The calculation and analysis process of the directed graph is that the client first initializes the system rule and writes multiple rules, performs related operations in the rule engine, and then judges the written rules through the detection module to determine whether they are compliant and legal. If yes, the written rule is converted into a directed graph, and it is detected whether the directed graph has a loop. Moreover, the initial topic and the last topic cannot be the same. If they are the same, it is considered to be annular, which will cause data explosion, and when it is found that there is a loop, the rule needs to be terminated immediately. If there is no loop, the rule is written into the cache and executed to perform real-time hit.The computing and analyzing module obtains the newly added rule data or the updated data in the cache module, and forms a directed acyclic graph of the topics with flow relationship for flow relationship analysis of the topics. When the relationship in the directed graph meets the "similarity" and the last flow data is a union set, the first coincidence point needs to be calculated for flow calculation at a specific time, and the data is calculated and obtained to flow to the next coincidence point, so as to repeatedly reduce the data flow to the downstream. The specific execution situation is returned to the client. The client sends events to the detection module for detection to determine the validity of the events. If the events are valid, the events enter the hit process, that is, the rules in the cache module are loaded into the memory, and the rules engine is run for real-time modification and restart. The corresponding rules engine of the corresponding topic is traversed, the event attribute and condition corresponding to the event are matched, it is judged whether to enter the similar graph, if yes, the other related events are waited to enter, otherwise the hit result is returned to the client as the hit information.

[0055] In the embodiment, a plurality of topics conforming to the user demand are created in advance, such as topic A, topic B, topic C, topic D, topic E and topic F. Then, all the created topics are checked. The checking method of the topics can be that a subscriber thread is started to subscribe to the topic, and a publisher thread is started to publish a message on the topic, and it is detected whether the subscriber can receive the message published by the publisher. If yes, it is determined that the topic is normal.

[0056] When it is determined that all the topics are normal, all the topics can be written into the nodes corresponding to the topics in sequence, such as writing all the topics into the respective blockchain nodes in sequence, and performing a blockchain node consensus operation to determine whether the topics are written into the respective blockchain nodes successfully. After the writing is successful, prompt information of writing success is output.

[0057] After the topics of the respective nodes are created, the rules of the respective nodes can be created according to the respective topics. Before the rules are created, the system rules are initialized, and the verified and available rules are moved to the memory. Then, the rules corresponding to the respective nodes are created. For example, rule A, rule B, rule C and rule D are created, as shown in Table 1.

[0058]

[0059] Table 1

[0060] And after the construction of each rule, the rules need to be detected, that is, to determine which rules in each rule are newly created rules at the current time or are modified rules at the current time, and the newly created rules and the modified rules are all new rules, at this time, all new rules need to be detected, the detection method is to first construct a directed graph according to the new rules, and then detect the loop of the directed graph to determine whether there is a loop in the directed graph, so as to avoid repeated messages circulating between several topics. And in this embodiment, the directed graph is constructed according to the new rules, that is, the topic flow relationship of the data source of the new rule and the topic to which the data flows (that is, the node flow direction) is determined first, and then the nodes of each topic in the new rule are connected according to the topic flow relationship. For example, as rule A in Table 1 above, the event is first obtained from topic A, the received event information is filtered, the attributes of the event are selected, and the information is forwarded to topic B, that is, a directed graph from vertex A (the node where the data source topic of the rule is located) to vertex B (the node where the topic to which the data flows is located) is formed.

[0061] And in this embodiment, when the directed graph is constructed, it can be first detected whether the directed graph is a connected graph, if it is a connected graph, a tree can be constructed according to each node and node flow direction in the directed graph, if a reverse edge appears in the process of constructing the tree, it is considered that there is a loop in the directed graph. If it is a non-connected graph, the different parts of the directed graph are respectively constructed into a tree to determine whether there is a loop in the directed graph. The specific process of constructing the tree can be to traverse all nodes in the directed graph from the first vertex in the directed graph, and put the traversed nodes into a recursive stack, if the newly traversed node already exists in the recursive stack, it is determined that there is a reverse edge, that is, it is determined that there is a loop in the directed graph.

[0062] Among them, the topics are the same data structure, belong to the same type of events belong to the same topic, and each topic has a queue of each other in logic. And each topic is completely isolated, and publishing, storing and notifying do not affect each other. The event is a message at the business level, including event ID, content, optional definition attribute (that is, event attribute, which is a field in a topic message) and associated topic. The rule is a pre-set description (such as rule name, rule flow and rule parameter, etc.), when the event is published, the rule matching is performed, and after the matching is successful, the flow is transferred according to the requirements of the rule. The connected graph is based on the probability of connectivity, in an undirected graph G, if there is a path connecting node i to node j, it is determined that node i and node j are connected. If G is a directed graph, and all edges in the path connecting node i and node j are in the same direction, if any two nodes in the directed graph are connected, it can be determined that the directed graph is a connected graph.

[0063] Step S20, if there is no loop in the directed graph, all events sent by the receiving client are received, and the target rule corresponding to each event is determined based on each rule;

[0064] When it is found that there is a loop in the directed graph constructed by the new rule through judgment, it is determined that the new rule set fails, and a prompt information of failure is returned. If there is no loop in the directed graph, it is determined that the new rule set is successfully constructed, and the new rule is stored. Then, all events sent by the receiving client are received, and each event is detected according to all rules corresponding to each topic, that is, each event is traversed, the event attribute of the traversed event is extracted, and the event attribute is matched with the topic in all rules. If there is a topic matching the event, the rule corresponding to the matching topic is taken as the target rule corresponding to the traversed event. And based on this, the target rule corresponding to all events is obtained. For example, when the client sends an event with event ID A to the node where topic A is located, and sends an event with event ID C to the node where topic C is located. And the content of A event contains ID A device, January 1, 2020, humidity 10, the event attribute of A event is extracted, that is, topic A {event ID: A, humidity: 10, date: January 1, 2020}. Similarly, if the content of C event contains ID C device, January 1, 2020, humidity 40, the event attribute of C event is extracted, that is, topic C {event ID: C, humidity: 40, date: January 1, 2020}. If A event is hit by rule A at this time, it is determined that the target rule corresponding to A event is rule A, and if C event is hit by rule C at this time, it is determined that the target rule corresponding to C event is rule C. Moreover, in this embodiment, after determining the target rule corresponding to each event, a preset threshold time (any time threshold set by the user in advance) is waited to detect whether there is a common sequence.

[0065] Moreover, in this embodiment, after obtaining each event, the event attribute is extracted, and the matching topic is determined according to the event attribute, that is, the associated topic can be as shown in Table 2.

[0066] Name Event ID Attribute Associated Topic Event A A Date, Humidity, Area Topic A Event B B Humidity, Temperature, Date, Area ID Topic B Event C C Temperature, Date, Area Topic C Event D D Area ID Topic D

[0067] Table 2

[0068] Step S30, according to the target rule corresponding to each event, the sequence of each event is determined, and whether there is a common sequence in each sequence is detected;

[0069] After determining the target rules corresponding to each event, the sequence of each event needs to be determined in turn according to each target rule, that is, the nodes and node flow direction through which each event flows. For example, if the target rule corresponding to event A is rule A, event A can be judged according to the directed graph of rule A to determine all nodes through which event A flows. That is, if the node flow direction of rule A is node A-node B-node D, if the event attribute of event A meets the subject corresponding to node A, meets the subject corresponding to node B, and meets the subject corresponding to node C, the sequence of event A can be directly determined as A-B-D, and the node flow direction relationship of event A is node A-node B-node D.

[0070] After obtaining the sequence of each event according to the target rule corresponding to each event, it can be determined whether there is a common sequence in each sequence, and different operations are performed according to different detection results. Among them, the common sequence is at least two nodes that are the same and the node flow direction in the two nodes is the same. For example, if the sequence corresponding to event A is A-B-D, and the node flow direction is node A-node B-node D; if the sequence corresponding to event C is C-B-D, and the node flow direction is node C-node B-node D; if the sequence corresponding to event D is D-B-C, and the node flow direction is node D-node B-node C; it is determined that there is a common sequence between event A and event C, and the common sequence is B-D. For example, as shown in Figure 5 If the sequence of event A is A-B-D and the sequence of event C is C-B-D, it is determined that the common sequence of event A and event C is B-D.

[0071] Step S40, if there is a common sequence, determining all target events corresponding to the common sequence, and merging the data of each target event in the common sequence.

[0072] When it is found through judgment that there is a common sequence, all target events corresponding to the common sequence need to be determined first. If the target events are event A and event C, the event attribute and event content of event A need to be obtained, and the event attribute and event content of event C need to be obtained. The repeated content in the event content of event A and the event content of event C is calculated, and the repeated content is merged. The repeated event attribute in the event attribute of event A and the event attribute of event C is calculated, and the repeated event attribute is merged, so as to complete the merging of the data of each target event in the common sequence. After merging the data in the common sequence, if there are still nodes to be flowed to in the subsequent nodes, the merged data is flowed to the subsequent nodes, so as to reduce the repeated data flowed to the subsequent nodes.

[0073] In addition, in order to assist understanding of the data processing in the embodiment, the following is an example.

[0074] For example, as shown in Figure 6As shown, when starting to process data, first, the preset rule is constructed, and then the directed graph is constructed according to the preset rule, and it is judged whether there is a loop in the directed graph, if yes, the user is informed, and the construction of the preset rule is terminated. If no, it is determined that the construction of the directed graph is successful, and after receiving all events sent by the client and determining the target rules corresponding to each event, the topology sequence of each event is analyzed to determine the sequence corresponding to each event, and it is detected whether there is a common sequence in each sequence, if no, the execution of each event is continued. If there is a common sequence, parameter algebra analysis of the event is performed, and according to the preset time threshold, the event convergence of the common sequence is performed, and it is written into the memory storage until the end.

[0075] In the embodiment, all rules corresponding to each node are determined according to the theme in all nodes, a directed graph is constructed based on new rules in each rule, and it is detected whether there is a loop in the directed graph; if there is no loop in the directed graph, all events sent by the client are received, and target rules corresponding to each event are determined based on each rule; the sequence of each event is determined according to the target rule corresponding to each event, and it is detected whether there is a common sequence in each sequence; if there is a common sequence, all target events corresponding to the common sequence are determined, and the data of each target event in the common sequence is merged. By determining all rules according to the theme in all nodes, and constructing a directed graph according to new rules in each rule, when there is no loop in the directed graph, target rules corresponding to all events sent by the client are determined, so as to determine the sequence of all events, and when there is a common sequence in each sequence, the data of all target events corresponding to the common sequence in the common sequence is merged, thereby avoiding the phenomenon that the repetitive message or the content overlapping instruction flow to the downstream node in the prior art, causing resource waste, reducing the probability of repetitive event flow to the downstream node, and avoiding resource waste.

[0076] Further, based on the first embodiment of the application, a second embodiment of the data processing method of the application is proposed, and the refinement of the step S30 in the above embodiment, the step of detecting whether there is a common sequence in each sequence, includes:

[0077] Step a, determining all nodes of a target sequence in each sequence, and matching all nodes of the target sequence with nodes of all sequences except the target sequence in each sequence in turn;

[0078] In the embodiment, after the sequences of the respective events are acquired, the nodes in the respective sequences need to be compared in sequence, that is, the target sequence in the respective sequences can be determined first, and all the nodes of the target sequence are extracted, and then all the nodes of the target sequence are matched with the nodes of all the sequences except the target sequence. The target sequence can be any sequence in the respective sequences.

[0079] Step b, if the first node in the target sequence and the second node in all the other sequences match, determining whether the flow direction nodes corresponding to the first node and the second node are the same;

[0080] Step c, detecting whether there is a common sequence in the respective sequences according to the determination result of whether the flow direction nodes corresponding to the first node and the second node are the same.

[0081] When there is a certain node (i.e. the first node) in the target sequence and a node (i.e. the second node) in all the other sequences, that is, the first node and the second node match, the node flow direction of the first node needs to be determined, and the next node to which the first node needs to flow, that is, the flow direction node corresponding to the first node, is determined according to the node flow direction. At the same time, the sequence corresponding to the second node also needs to be determined, and the node flow direction of the second node is determined according to the sequence, so as to determine the next node to which the second node needs to flow, that is, the flow direction node corresponding to the second node. When the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are the same, it can be directly determined that there is a common sequence in the respective sequences, and the common sequence is from the first node (or the second node) to the flow direction node corresponding to the first node (or the flow direction node corresponding to the second node). However, if the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are not the same, the subsequent nodes need to be detected to determine whether there is a common sequence.

[0082] In the embodiment, by matching all the nodes of the target sequence in the respective sequences with the nodes of all the other sequences, if the first node of the target sequence and the second node of all the other sequences match, and whether there is a common sequence in the respective sequences is determined according to whether the flow direction node of the first node and the flow direction node of the second node are the same, the accuracy of detecting the common sequence is improved.

[0083] Further, the step of merging the data of the target events in the common sequence includes:

[0084] Step d, determining the event attribute and event content of the target events in the common sequence based on a preset threshold time;

[0085] In the embodiment, when it is determined that there is a common sequence and all target events corresponding to the common sequence are determined, because the time nodes of the target events to the common sequence are different, a threshold time can be set first, and the event attribute and event content of each target event transmitted to the start point (i.e., the first node in the common sequence) of the common sequence are acquired within the threshold time.

[0086] Step e, merging the repeated contents in each of the event contents and merging the repeated event attributes in each of the event attributes.

[0087] When the event attribute and event content corresponding to each target event are acquired, the repeated contents in each of the event contents can be merged to obtain merged event contents, and the repeated event attributes in each of the event attributes can be merged to obtain merged event attributes. If the common sequence still needs to flow to a downstream node, the merged event contents and merged event attributes can be directly transmitted to the downstream node.

[0088] In the embodiment, the repeated event attributes in the event attributes of each target event in the common sequence are merged and the repeated contents in the event contents in the common sequence are merged according to the preset threshold time, so that the effective performance of the merging process is ensured.

[0089] Further, the step of constructing a directed graph based on the new rules in each of the rules comprises:

[0090] Step f, determining a target node corresponding to a new rule in each of the rules based on each of the nodes, and taking the theme in the target node as a target theme;

[0091] In the embodiment, after the rules corresponding to each theme are determined, each rule needs to be detected, that is, it is determined which rules in each rule are newly created rules at the current time or are rules modified at the current time, and the newly created rules and the modified rules are all taken as new rules. At this time, all the new rules need to be detected, that is, the node to which the new rule is applied can be determined first and taken as a target node, and the theme in the target node is taken as a target theme.

[0092] Step g, determining the node flow direction of the new rule in each of the target nodes according to each of the target themes, connecting each of the target nodes according to the node flow direction, and taking the connected target nodes as a directed graph.

[0093] When the target topics of the new rules are acquired, the node flow directions of the new rules at the target nodes are determined according to the requirements in the target topics, and the target nodes are connected according to the node flow directions, and the graph formed by the connected target nodes is taken as a directed graph. For example, as the rule A in Table 1, the events are acquired from the topic A first, the received event information is filtered, the attributes of the events are selected, and the information is forwarded to the topic B, so as to form a directed graph with vertex A (the node where the data source of the rule is located) pointing to vertex B (the node where the flow direction is located).

[0094] In the embodiment, the target nodes corresponding to the new rules are determined in each rule, the node flow directions are determined according to the topics in the target nodes, all the target nodes are connected according to the node flow directions, and the directed graph is determined, so as to guarantee the accuracy of the acquired directed graph.

[0095] Further, the step of detecting whether there is a loop in the directed graph comprises:

[0096] Step h, determining all the target nodes in the directed graph, determining the vertex in each target node according to the node flow direction of each target node, and traversing each target node from the vertex according to each node flow direction;

[0097] In the embodiment, when the directed graph is constructed, the directed graph needs to be detected, that is, all the target nodes contained in the directed graph are acquired, the node flow directions of the target nodes are determined, the vertex (that is, the first node running and flowing in the directed graph) of each target node is determined according to the node flow direction, and then each target node is traversed from the vertex according to each node flow direction, so as to determine whether there is a loop in the directed graph.

[0098] Step k, matching the traversed target node and other nodes in each target node except the traversed target node, and determining whether there is a loop in the directed graph based on the matching result of the matching.

[0099] The traversed target node and other nodes in each target node except the traversed target node are matched, if there is a target node matching (that is, the same) the traversed target node, it is determined that there is a loop in the directed graph, if there is no target node matching (that is, not the same) the traversed target node, it is determined that there is no loop in the directed graph. And the specific way to check whether there is a loop in the directed graph can be to construct a tree for the directed graph, and to determine whether there is a loop in the directed graph by determining whether there is a reverse edge.

[0100] In the embodiment, the vertex is determined according to the node flow direction of all target nodes in the directed graph, each target node is traversed from the vertex according to the node flow direction, the traversed target node and other nodes are matched, it is determined whether there is a loop in the directed graph based on the matching result of the matching, and therefore the accuracy of detection is ensured.

[0101] Further, the step of determining the target rule corresponding to each event based on the rules comprises:

[0102] Step m: sequentially traversing each event, obtaining the event attribute of the traversed event, and matching the event attribute with each rule;

[0103] In the embodiment, when the target rule corresponding to each event is determined, each event can be traversed, the event attribute of the traversed event is extracted, and the event attribute is matched with the theme in all rules. If there is a theme matched with the event, the rule corresponding to the matched theme is taken as the target rule corresponding to the traversed event. For example, when the client sends an event with an event ID A to a node where theme A is located, and sends an event with event C to a node where theme C is located. Moreover, the A event content contains an ID A device, January 1, 2020, and the humidity is 10. The event attribute of the A event is extracted, that is, theme A {event ID: A, humidity: 10, date: January 1, 2020}. Similarly, if the C event content contains an ID C device, January 1, 2020, and the humidity is 40, the event attribute of the A event is extracted, that is, theme C {event ID: C, humidity: 40, date: January 1, 2020}. If the A event is hit by rule A at this time, it is determined that the target rule corresponding to the A event is rule A. If the C event is hit by rule C at this time, it is determined that the target rule corresponding to the C event is rule C.

[0104] Step n: if there is a matching rule matched with the event attribute in each rule, the matching rule is taken as the target rule corresponding to the traversed event.

[0105] When it is found through judgment that there is a rule matched with the event attribute in each rule (that is, a matching rule), the matching rule can be directly taken as the target rule corresponding to the traversed event, and the target rule corresponding to the next traversed event is determined.

[0106] In the embodiment, the event attribute of the traversed event is matched with each rule by traversing each event, and the matching rule matched is taken as the target rule corresponding to the traversed event, so that the accuracy of the obtained target rule is ensured.

[0107] Further, before the step of determining all rules corresponding to each node based on the theme in all nodes, the method comprises:

[0108] Step x, obtaining a plurality of topics inputted, and detecting whether each of the topics meets preset node requirements in sequence;

[0109] Step y, if each of the topics meets the preset node requirements, determining a node corresponding to each of the topics, and assigning each of the topics to the node corresponding to each of the topics in sequence.

[0110] In the embodiment, before determining the rules corresponding to each node, a plurality of topics conforming to user requirements, such as topic A, topic B, topic C, topic D, topic E and topic F, are created in advance. Then, detection is performed on all the created topics to determine whether each of the topics meets preset node requirements, that is, a subscriber thread can be started to subscribe to the topic, and a publisher thread can be started to publish a message on the topic, and it is detected whether the subscriber can receive the message published by the publisher. If yes, it is determined that the topic is normal and meets the node requirements. When all the topics meet the preset node requirements, all the topics can be written into the nodes corresponding to the topics in sequence, such as writing all the topics into the respective blockchain nodes, and performing a blockchain node consensus operation to determine whether the topics in each of the blockchain nodes are written successfully, and after the writing is successful, prompt information of writing success is outputted.

[0111] In the embodiment, by obtaining a plurality of topics inputted, and assigning each of the topics to the node corresponding to each of the topics in sequence when each of the topics meets the node requirements, the effectiveness of the topics is ensured.

[0112] The application further provides a data processing apparatus, referring to Figure 3 , the data processing apparatus comprises:

[0113] A construction unit A10 is configured to determine all rules corresponding to each of the nodes according to the topics in all the nodes, construct a directed graph based on new rules in each of the rules, and detect whether there is a loop in the directed graph.

[0114] A determination unit A20 is configured to receive all events sent by a client if there is no loop in the directed graph, and determine target rules corresponding to each of the events based on each of the rules.

[0115] A detection unit A30 is configured to determine sequences of each of the events according to the target rules corresponding to each of the events, and detect whether there is a common sequence in each of the sequences.

[0116] A merging unit A40 is configured to determine all target events corresponding to the common sequence if there is the common sequence, and merge data of each of the target events in the common sequence.

[0117] Optionally, the detection unit A30 is configured to:

[0118] determining all nodes of a target sequence in each of the sequences, and matching all nodes of the target sequence with nodes of all other sequences in each of the sequences except the target sequence in turn;

[0119] if the first node in all nodes of the target sequence matches the second node in all other sequences, determining whether the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are the same;

[0120] detecting whether there is a common sequence in each of the sequences according to a determination result of the determination of whether the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are the same.

[0121] Optionally, the merging unit A40 is configured to:

[0122] determining event attributes and event contents of each of the target events in the common sequence based on a preset threshold time;

[0123] merging repeated contents in each of the event contents, and merging repeated event attributes in each of the event attributes.

[0124] Optionally, the constructing unit A10 is configured to:

[0125] determining a target node corresponding to a new rule in each of the rules based on each of the nodes, and taking a subject in the target node as a target subject;

[0126] determining node flow directions of the new rule in each of the target nodes according to each of the target subjects, connecting each of the target nodes according to the node flow directions, and taking the connected each of the target nodes as a directed graph.

[0127] Optionally, the constructing unit A10 is configured to:

[0128] determining all target nodes in the directed graph, determining a vertex in each of the target nodes according to a node flow direction of each of the target nodes, and traversing each of the target nodes from the vertex according to each of the node flow directions;

[0129] matching the traversed target node with other nodes in each of the target nodes except the traversed target node, and determining whether there is a loop in the directed graph based on a matching result of the matching.

[0130] Optionally, the determining unit A20 is configured to:

[0131] traversing each of the events in turn, obtaining an event attribute of a traversed event, and matching the event attribute with each of the rules;

[0132] If there is a matching rule matching the event attribute in each of the rules, the matching rule is taken as a target rule corresponding to the traversed event.

[0133] Optionally, the constructing unit A10 is configured to:

[0134] The plurality of topics inputted are acquired, and whether each of the topics meets preset node requirements is detected in sequence.

[0135] If each of the topics meets the preset node requirements, nodes corresponding to each of the topics are determined, and each of the topics is assigned to the node corresponding to the topic in sequence.

[0136] The method performed by each of the program units can refer to each of the embodiments of the data processing method, and will not be described here again.

[0137] The application further provides a computer storage medium.

[0138] The data processing program stored on the computer storage medium of the application is executed by a processor to realize the steps of the data processing method as described above.

[0139] The method realized by the data processing program executed by the processor can refer to each of the embodiments of the data processing method, and will not be described here again.

[0140] It should be noted that, in this document, the terms “comprising”, “including”, or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or system. Without more limitations, the element defined by the statement “including a…” does not exclude the presence of another identical element in the process, method, article, or system including the element.

[0141] The above-mentioned embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0142] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, can also be through hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art contribution can be embodied in the form of software products, the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc) as described above, including a number of instructions to make a terminal device (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0143] The above is only the preferred embodiment of the present application, not therefore limit the patent scope of the present application, any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A data processing method, characterized in that, The data processing method includes the following steps: Obtain multiple input topics and sequentially check whether each topic meets the preset node requirements; If each of the topics meets the preset node requirements, then the node corresponding to each topic is determined, and each topic is assigned to the node corresponding to each topic in sequence. Based on the themes in all nodes, determine all rules corresponding to each node, construct a directed graph based on the new rules in each of the rules, and detect whether there is a cycle in the directed graph; the new rules are newly created rules and / or modified rules; If there are no cycles in the directed graph, then all events sent by the client are received, and the target rule corresponding to each event is determined based on each of the rules. The sequence of each event is determined according to the target rule corresponding to each event, and the common sequence is detected among the sequences. If a common sequence exists, all target events corresponding to the common sequence are determined, and the event attributes and event content of each target event in the common sequence are determined based on a preset threshold time. Duplicate content in each event content is merged, and duplicate event attributes in each event attribute are merged; if there are subsequent nodes that need to be flowed, the merged data is flowed to the subsequent nodes.

2. The data processing method as described in claim 1, characterized in that, The step of detecting whether there is a common sequence among the sequences includes: Determine all nodes of the target sequence in each of the sequences, and match all nodes of the target sequence with nodes of all other sequences in each of the sequences except the target sequence in turn; If a first node in all nodes of the target sequence matches a second node in all other sequences, then determine whether the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are the same. Based on the determination result of whether the flow direction node corresponding to the first node and the flow direction node corresponding to the second node are the same, it is detected whether there is a common sequence in each sequence.

3. The data processing method as described in claim 1, characterized in that, The step of constructing a directed graph based on the new rules among the aforementioned rules includes: Based on each of the nodes, the target node corresponding to the new rule in each of the rules is determined, and the topic in the target node is taken as the target topic; The new rule is determined at each of the target nodes according to the target themes, and the target nodes are connected according to the node flow, forming a directed graph.

4. The data processing method as described in claim 1, characterized in that, The step of detecting whether a cycle exists in the directed graph includes: All target nodes in the directed graph are determined, the vertices in each target node are determined according to the node flow direction of each target node, and each target node is traversed starting from the vertex according to the node flow direction. The target node to be traversed is matched with other nodes in the target node other than the target node to be traversed, and the existence of a cycle in the directed graph is determined based on the matching results.

5. The data processing method as described in claim 1, characterized in that, The step of determining the target rule corresponding to each event based on each of the rules includes: Each of the events is traversed sequentially, the event attributes of the traversed events are obtained, and the event attributes are matched with each of the rules. If a matching rule exists among the rules that matches the event attribute, then the matching rule is taken as the target rule corresponding to the traversed event.

6. A data processing apparatus, characterized in that, The data processing device includes: A construction unit is used to acquire multiple input topics and sequentially detect whether each topic meets preset node requirements; if each topic meets the preset node requirements, the node corresponding to each topic is determined and each topic is sequentially assigned to the node corresponding to each topic. A construction unit is used to determine all rules corresponding to each node based on the topics in all nodes, construct a directed graph based on the new rules in each of the rules, and detect whether there is a cycle in the directed graph; the new rules are newly created rules and / or modified rules; A determining unit is configured to determine that the newly set rule fails if a cycle exists in the directed graph. The determining unit is configured to receive all events sent by the client if there is no cycle in the directed graph, and determine the target rule corresponding to each event based on each rule. The detection unit is used to determine the sequence of each event according to the target rule corresponding to each event, and to detect whether there is a common sequence in each sequence; The merging unit is used to determine all target events corresponding to the common sequence if a common sequence exists, and to determine the event attributes and event content of each target event in the common sequence based on a preset threshold time; merge the duplicate content in each event content, and merge the duplicate event attributes in each event attribute; if there are still nodes that need to be flowed to, then the merged data is flowed to the subsequent nodes.

7. A data processing device, characterized in that, The data processing device includes: a memory, a processor, and a data processing program stored in the memory and executable on the processor, wherein the data processing program, when executed by the processor, implements the steps of the data processing method as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The computer storage medium stores a data processing program, which, when executed by a processor, implements the steps of the data processing method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product stores readable instructions, which, when read and executed by a computer, cause the computer to perform the steps of the data processing method as described in any one of claims 1 to 5.

Images