A distributed event hub networking method
By acquiring and analyzing node operating status data in a distributed network, calculating event similarity and allocation adjustment coefficients, and optimizing node allocation, the problem of load imbalance is solved and event processing and response speed is improved.
Patent Information
- Application Number
- CN202411139599.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The existing node allocation method leads to load imbalance in a distributed network environment, reducing event processing speed and system response speed, and thus reducing the allocation effect of new events.
By obtaining the operating status data of unassigned nodes and historical nodes of the distributed network hub system, the similarity and allocation adjustment coefficients between events are calculated, the initial similarity is corrected, and the node allocation is optimized to achieve load balancing and response speed improvement.
The node allocation effect of distributed systems on newly added events is improved, the load balancing and response speed is ensured, and the network environment is adapted to dynamically changing.
Smart Images

Figure CN118659979B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of network node allocation, and in particular to a distributed event hub networking method. Background Art
[0002] Distributed network systems are an important means of processing massive amounts of data and high-concurrency requests. Among them, distributed event center networking systems are used to monitor, analyze and respond to various events. Such systems distribute tasks among multiple processing nodes to achieve efficient event processing and system load balancing, as well as efficient event processing and data distribution. This not only improves the flexibility and scalability of the network, but also enhances the reliability and response speed of the entire system.
[0003] In the related technology, static configuration or simple load balancing strategies are usually used to allocate nodes for various events generated by distributed systems. However, due to the complexity of changes in distributed network environments and certain similarities between different events, only using existing node allocation methods will lead to unbalanced load in distributed systems, reduce event processing speed and system response speed, and thus reduce the effect of node allocation for new events. Summary of the invention
[0004] In order to solve the technical problem that only using the existing node allocation method will lead to unbalanced load of the distributed system, reduce the event processing speed and the system response speed, and thus reduce the effect of node allocation for new events, the purpose of the present invention is to provide a distributed event hub networking method, and the technical solution adopted is as follows:
[0005] The present invention proposes a distributed event hub networking method, the method comprising:
[0006] Obtain new events of unassigned nodes in the distributed network central system at the current moment, and historical events of assigned nodes at each historical moment in a preset time period, and at the same time obtain the operating status data of each node in different dimensions at the current moment and each historical moment, wherein both new events and historical events include basic event data of different fields;
[0007] Taking any historical event as a target historical event, obtaining the initial similarity between the newly added event and the target historical event according to the difference of the event basic data of the same field between the newly added event and the target historical event;
[0008] The preset time period is divided into multiple short-term windows, and the allocation adjustment coefficient of the new event is obtained according to the operating status data of all nodes in each dimension at each historical moment in the short-term window, and the operating status data of all nodes in each dimension at the current moment; the node assigned to the target historical event is taken as the target node, and the historical moment where the target historical event is located is taken as the target historical moment, and the operating status difference of the target node at the current moment and the target historical moment is obtained according to the difference of the operating status data of the target node in the same dimension between the current moment and the target historical moment;
[0009] According to the allocation adjustment coefficient and the running state difference, the initial similarity is corrected to obtain the real similarity between the new event and the target historical event; based on the real similarity between the new event and each historical event in each node, the new event is allocated to a node.
[0010] Furthermore, obtaining the initial similarity between the newly added event and the target historical event according to the difference of the event basic data of the same field between the newly added event and the target historical event includes:
[0011] Taking any field as the target field, normalizing the information entropy of the event basic data of the newly added event and all historical events in the target field to obtain a first weight parameter of the target field, wherein the sum of the first weight parameters of all fields is equal to a value of 1;
[0012] Performing negative correlation normalization processing on the difference of the event basic data between the newly added event and the target historical event in the target field to obtain the data similarity between the newly added event and the target historical event with respect to the target field;
[0013] Based on the first weight parameter of each field, a weighted sum is performed on the data similarity between the newly added event and the target historical event in each field to obtain an initial similarity between the newly added event and the target historical event.
[0014] Further, obtaining the allocation adjustment coefficient of the newly added event according to the operation status data of each dimension of all nodes at each historical moment in the short-time window and the operation status data of each dimension of all nodes at the current moment includes:
[0015] Taking any dimension of the node as the target dimension, normalizing the information entropy of the operating status data of the target dimension of all nodes at all historical moments to obtain a second weight parameter of the target dimension, wherein the sum of the second weight parameters of all dimensions of the node is equal to a value of 1;
[0016] Taking any short-time window as a target short-time window, analyzing the overall level of the operating status data of the target dimension of all nodes at all historical moments in the target short-time window, and obtaining the load parameter of the distributed network with respect to the target dimension in the target short-time window;
[0017] Based on the second weight parameter of each dimension, weighted sum is performed on the load parameter of each dimension in the target short-time window to obtain the overall load degree of the distributed network in the target short-time window;
[0018] Obtaining a load threshold of the distributed network according to the overall load levels of all short-term windows;
[0019] The distribution adjustment coefficient of the new event is obtained based on the operating status data of the target dimension of all nodes at the current moment, the second weight parameter of the target dimension, and the load threshold of the distributed network.
[0020] Further, obtaining a load threshold of the distributed network according to the overall load degree of all short-term windows includes:
[0021] The upper quartile of the overall load level of all short-term windows is used as the load threshold of the distributed network.
[0022] Further, obtaining the distribution adjustment coefficient of the newly added event according to the operating status data of the target dimension of all nodes at the current moment, the second weight parameter of the target dimension, and the load threshold of the distributed network includes:
[0023] Analyze the overall level of the operating status data of all nodes in the target dimension at the current moment to obtain the load factor of the distributed network with respect to the target dimension at the current moment;
[0024] Based on the second weight parameter of each dimension, weighted summation is performed on the load factor of each dimension at the current moment to obtain the overall load index of the distributed network at the current moment;
[0025] When the overall load index of the distributed network at the current moment is greater than the load threshold, normalizing the difference between the overall load index of the distributed network at the current moment and the load threshold to obtain a distribution adjustment coefficient of the newly added event;
[0026] When the overall load index of the distributed network at the current moment is not greater than the load threshold, the allocation adjustment coefficient of the newly added event is set to 0.
[0027] Further, obtaining the difference in the running state of the target node at the current moment and the target historical moment according to the difference in the running state data of the same dimension between the target node at the current moment and the target historical moment includes:
[0028] According to the difference of the running state data of the target dimension between the current moment and the target historical moment of the target node, a state difference parameter of the target dimension between the current moment and the target historical moment of the target node is obtained;
[0029] Based on the second weight parameter of each dimension, a weighted sum is performed on the state difference parameter of the target node at the current moment and the target historical moment with respect to each dimension to obtain the running state difference of the target node at the current moment and the target historical moment.
[0030] Furthermore, the initial similarity is corrected according to the allocation adjustment coefficient and the running state difference to obtain the real similarity between the newly added event and the target historical event, including:
[0031] Performing negative correlation normalization processing on the running state difference to obtain the running state similarity of the target node at the current moment and the target historical moment;
[0032] The product value of the allocation adjustment coefficient and the operating state similarity is used as an adjustment parameter; the product value of the initial similarity and the adjustment parameter is used as an adjustment amount;
[0033] The sum of the initial similarity and the adjustment amount is used as the true similarity between the newly added event and the target historical event.
[0034] Furthermore, the node allocation for the new event based on the real similarity between the new event and each historical event in each node includes:
[0035] According to the true similarity between the newly added event and each historical event in each node, the nodes to be allocated and the priority allocation degree of each node to be allocated are obtained;
[0036] Based on the priority allocation degree of each node to be allocated, node allocation is performed on the newly added event.
[0037] Furthermore, obtaining the nodes to be allocated and the priority allocation degree of each node to be allocated according to the real similarity between the newly added event and each historical event in each node includes:
[0038] Selecting a preset number of historical events corresponding to the largest true similarities as similar historical events, and taking the nodes where the similar historical events are located as nodes to be allocated;
[0039] The overall level of the true similarity between the newly added event and all similar historical events in each node to be assigned is analyzed to obtain the priority assignment degree of each node to be assigned.
[0040] Furthermore, the node allocation for the newly added event based on the priority allocation degree of each node to be allocated includes:
[0041] Assign the newly added events to the to-be-assigned node corresponding to the maximum priority assignment degree.
[0042] The present invention has the following beneficial effects:
[0043] The present invention takes into account the complexity of changes in distributed network environments and certain similarities between different events. Using only the existing technology will lead to an unbalanced load on the distributed system, reduce the event processing speed and the system's response speed, and further reduce the effect of node allocation for new events. Therefore, firstly, new events to which no nodes are allocated in the distributed network central system at the current moment and historical events to which nodes are allocated at each historical moment in a preset time period are obtained. At the same time, the operating status data of each node in different dimensions at the current moment and each historical moment are obtained. Then, the similarity between the new event and the target historical event is preliminarily reflected through the obtained initial similarity. Considering that node allocation for the new event only based on the initial similarity will cause excessive load on some nodes, the initial similarity between the new event and the target historical event is adjusted in the subsequent process through the obtained allocation adjustment coefficient. The similarity is adjusted to ensure the load balancing of the distributed network system after nodes are assigned to the newly added events. Since the distributed network environment is in complex dynamic changes during the event allocation process, there are large differences in the operating states of different nodes. It is necessary to add the node operating state to the consideration factors of event similarity measurement. Therefore, the difference in the operating state of the node where the target historical event is located at the current moment and the target historical moment can be reflected by obtaining the operating state difference. Subsequently, the initial similarity of the newly added event and the target historical event can be adjusted based on the operating state difference, so as to improve the response speed of the distributed system to the newly added event after the node is assigned. Then, based on the allocation adjustment coefficient and the operating state difference, the initial similarity is corrected, and the similarity between the newly added event and the historical event is accurately reflected by the obtained real similarity, thereby improving the effect of subsequent node allocation for the newly added event. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 A flow chart of a distributed event hub networking method provided by an embodiment of the present invention;
[0046] Figure 2 A flow chart of a method for obtaining a distribution adjustment coefficient of a newly added event provided by an embodiment of the present invention;
[0047] Figure 3 A flow chart of a method for assigning nodes to newly added events provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0048] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of the specific implementation method, structure, characteristics and effects of a distributed event hub networking method proposed in accordance with the present invention, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0050] The specific scheme of a distributed event hub networking method provided by the present invention is described in detail below with reference to the accompanying drawings.
[0051] See also Figure 1 , which shows a flow chart of a distributed event hub networking method provided by an embodiment of the present invention, the method comprising:
[0052] Step S1: Obtain new events of unassigned nodes in the distributed network central system at the current moment, and historical events of assigned nodes at each historical moment in a preset time period, and at the same time obtain the operating status data of each node in different dimensions at the current moment and each historical moment. Both new events and historical events include basic event data of different fields.
[0053] Distributed network systems are an important means of processing massive amounts of data and high-concurrency requests. Among them, distributed event hub networking systems are used to monitor, analyze and respond to various events. Such systems distribute tasks among multiple processing nodes to achieve efficient event processing and system load balancing, as well as efficient event processing and data distribution. This not only improves the flexibility and scalability of the network, but also enhances the reliability and response speed of the entire system. Related technologies usually use static configuration or simple load balancing strategies to allocate nodes to various events generated by distributed systems. However, due to the complexity of changes in distributed network environments and certain similarities between different events, only using existing node allocation methods will lead to unbalanced load on the distributed system, reduce the event processing speed and system response speed, and thereby reduce the effect of node allocation for new events. Therefore, an embodiment of the present invention proposes a distributed event hub networking method to solve this problem.
[0054] When various requests sent by the client through the network reach the server, the server system will generate various events to process the client's requests, and the event central system in the distributed network will assign the various generated events to nodes in the network, namely, servers, to realize rapid processing of events. Therefore, the embodiment of the present invention first obtains the newly added events of the distributed network central system that are not assigned to nodes at the current moment, and the historical events that are assigned to nodes at each historical moment in a preset time period, wherein the newly added events and historical events include basic event data of multiple different fields for recording basic information of the event, for example, each field includes at least a timestamp, an event type, and an event content, etc., which are not limited here. The timestamp records the time when the event occurs. Since the distributed network central system assigns events very quickly, in the embodiment of the present invention, the timestamp of the newly added event is considered to be the current moment, and the timestamp of each historical event is the historical moment corresponding to the historical event. Subsequently, based on the difference in the basic event data of the same field, a preliminary analysis can be performed on the similarity between the newly added event and the historical event, wherein the preset time period is set to 24 hours, and the specific value of the preset time period can also be set by the implementer according to the specific implementation scenario, which is not limited here.
[0055] Taking into account that the distributed network environment is in complex dynamic changes, there are large differences in the operating status of each node in the network at different times. If the differences in the operating status of the nodes are not taken into account, after the nodes are assigned to the new events, it will cause an unbalanced load on the distributed network system, thereby reducing the effect of node allocation for the new events. Therefore, it is also necessary to obtain the operating status data of each node in different dimensions at the current moment and each historical moment, where each dimension includes at least CPU utilization, memory utilization, network bandwidth utilization, disk I / O utilization, etc., which are not limited here. Subsequently, the similarity between the new events and historical events can be analyzed based on the node's operating status data to improve the effect of node allocation for the new events.
[0056] Step S2: taking any historical event as a target historical event, and obtaining the initial similarity between the newly added event and the target historical event according to the difference in event basic data of the same field between the newly added event and the target historical event.
[0057] Since the client usually accesses the same or similar data information through the network, there are a large number of similar requests in all the requests sent to the server. Therefore, some events generated in the distributed network system also have similar characteristics. Assigning new events to nodes corresponding to historical events with higher similarity for processing can reduce the number of times data is transmitted between nodes, thereby reducing network latency and improving response speed. The difference in basic event data of the same field between events with higher similarity is smaller. Therefore, an embodiment of the present invention first takes any historical event as the target historical event, and analyzes the difference in basic event data of the same field between the new event and the target historical event. The similarity between the new event and the target historical event is preliminarily evaluated by obtaining the initial similarity. The initial similarity can be further adjusted later to improve the accuracy of the similarity evaluation between the new event and the target historical event.
[0058] Preferably, in one embodiment of the present invention, the method for obtaining the initial similarity between the newly added event and the target historical event specifically includes:
[0059] Take any field as the target field, for example, the timestamp field can be taken as the target field, and the information entropy of the basic data of the new events and all historical events in the target field is normalized to obtain the first weight parameter of the target field. The first weight parameter can be used to weight the relevant data later to improve the accuracy of the initial similarity calculation and analysis. The method of calculating the information entropy is a technical means well known to those skilled in the art and will not be elaborated here.
[0060] In one embodiment of the present invention, the information entropy of the target field is used as the numerator, the sum of the information entropies of all fields is used as the denominator, and the ratio is used as the first weight parameter of the target field, so as to realize the normalization of the information entropy of the event basic data of new events and all historical events in the target field, and at the same time ensure that the sum of the first weight parameters of all fields is equal to the value 1, which is convenient for subsequent calculations.
[0061] The smaller the difference in the basic data of the new event and the target historical event in the same field, the greater the similarity between the new event and the target historical event. Therefore, the difference in the basic data of the new event and the target historical event in the target field is normalized to obtain the data similarity between the new event and the target historical event on the target field.
[0062] In an embodiment of the present invention, if the target field is a timestamp field, the event basic data of this field records the time information of the corresponding event. Then, the absolute value of the difference between the timestamps of the newly added event and the target historical event can be calculated as the difference in the timestamp fields between the two. If the target field is an event type field, the event basic data of this field records the processing type information of the corresponding event. If the event types of the newly added event and the target historical event are the same, the difference in the event type fields between the two is set to 1, otherwise the difference in the event type fields between the two is set to a value less than 1 and greater than 0, for example, it can be set to 0.5. If the target field is an event content field, the event basic data of this field records the specific content information of the corresponding event. Then, the edit distance of the event content field between the newly added event and the target historical event can be used as the difference in the event content field between the two. The edit distance is a technical means well known to those skilled in the art and will not be elaborated here. If the target field is other fields, the differences in other fields between the newly added event and the target historical event can also be analyzed in a similar manner, which will not be elaborated here.
[0063] In the embodiment of the present invention, the natural function A negative exponential function with base The negative correlation is normalized in the form of a function, which is not limited here, where Represents the normalization function, and the same method can be used in subsequent steps to achieve negative correlation normalization.
[0064] Through the above method, the first weight parameter of each field and the data similarity between the newly added event and the target historical event with respect to each field can be obtained.
[0065] The greater the data similarity between the new event and the target historical event in each field, the more similar the information recorded by the new event and the target historical event about the field is. At the same time, the larger the first weight parameter of each field, the higher the confidence that the information recorded by the new event and the target historical event about the field is similar. Therefore, based on the first weight parameter of each field, the data similarity between the new event and the target historical event in each field can be weighted and summed to obtain the initial similarity between the new event and the target historical event.
[0066] The expression of the initial similarity between the newly added event and the target historical event can be specifically, for example, as follows:
[0067]
[0068] in, Indicates the initial similarity between the newly added event and the target historical event; Indicates new events and all historical events about The information entropy of the event basic data of each field; Indicates new events and all historical events about The information entropy of the basic data of events in each field; Indicates that the newly added event and the target historical event are in The difference in event basic data between fields; Indicates the number of fields; Indicates The first weight parameter of the field; Indicates the newly added events and the target historical events about The data similarity of the fields; Represented by natural function An exponential function with base .
[0069] At this point, the initial similarity between the newly added event and the target historical event is obtained.
[0070] Step S3: Divide the preset time period into multiple short-term windows, and obtain the allocation adjustment coefficient of the new event based on the operating status data of each dimension of all nodes at each historical moment in the short-term window, and the operating status data of each dimension of all nodes at the current moment; take the node assigned to the target historical event as the target node, and the historical moment where the target historical event is located as the target historical moment, and obtain the operating status difference of the target node at the current moment and the target historical moment based on the difference in the operating status data of the same dimension between the target node at the current moment and the target historical moment.
[0071] Taking into account certain special periods within the preset time period, the distributed network system has to process a large number of events, resulting in excessive load on some nodes in the distributed network system. If the nodes of the newly added events are allocated only according to the initial similarity, the load of the nodes will be aggravated, causing the load of the distributed network system to be unbalanced. Therefore, the embodiment of the present invention first divides the preset time period into multiple short-term windows, and then analyzes the overall load of the system in each short-term window based on the operating status data of all nodes in each dimension at each historical moment in the short-term window. At the same time, based on the operating status data of all nodes in each dimension at the current moment, the overall load of the system at the current moment is analyzed, and then the initial similarity between the newly added event and the target historical event is adjusted in the subsequent step through the obtained allocation adjustment coefficient to ensure load balancing of the distributed network system after nodes are allocated to the newly added events.
[0072] Preferably, in one embodiment of the present invention, a method for obtaining a distribution adjustment coefficient of a newly added event specifically includes:
[0073] See also Figure 2 , which shows a flow chart of a method for obtaining a distribution adjustment coefficient of a new event provided by an embodiment of the present invention.
[0074] Step S301: Take any dimension of the node as the target dimension, normalize the information entropy of the operating status data of the target dimension of all nodes at all historical moments, and obtain the second weight parameter of the target dimension, wherein the sum of the second weight parameters of all dimensions of the node is equal to the value 1.
[0075] First, any dimension of the node is taken as the target dimension, for example, the CPU usage dimension can be taken as the target dimension, and the information entropy of the operating status data of the target dimension of all nodes at all historical moments is normalized to obtain the second weight parameter of the target dimension. The second weight parameter can be used to weight the relevant data later to improve the accuracy of calculation and analysis.
[0076] In one embodiment of the present invention, the information entropy of the target dimension is used as the numerator, the sum of the information entropies of all dimensions is used as the denominator, and the ratio is used as the second weight parameter of the target dimension, so as to realize the normalization of the information entropy of the operating status data of the target dimension of all nodes at all historical moments, while ensuring that the sum of the second weight parameters of all dimensions is equal to the value 1, which is convenient for subsequent calculations.
[0077] The second weight parameter of each dimension can be obtained by the same method as above.
[0078] Step S302: Take any short-time window as the target short-time window, analyze the overall level of the operating status data of the target dimension of all nodes at all historical moments in the target short-time window, and obtain the load parameters of the distributed network in the target dimension in the target short-time window.
[0079] Since the load conditions of the distributed network system in different short-time windows are different, any short-time window can be analyzed. First, any short-time window is taken as the target short-time window. The larger the operating status data of the target dimension of all nodes at all times in the target short-time window, the greater the load of the distributed network system on the target dimension in the target short-time window. For example, the greater the CPU usage of all nodes at all times in the target short-time window, the greater the CPU usage load of the distributed network system in the target short-time window. Therefore, the overall level of the operating status data of the target dimension of all nodes at all historical moments in the target short-time window can be analyzed. The obtained load parameters reflect the load condition of the distributed network system on the target dimension in the target short-time window, providing a data basis for the subsequent analysis of the overall load condition of the distributed network in the target short-time window.
[0080] In an embodiment of the present invention, the average value or median of the operating status data of the target dimension of all nodes at all historical moments in the target short-time window can be used as the load parameter of the distributed network with respect to the target dimension in the target short-time window to realize the analysis of its overall level, which is not limited here.
[0081] The load parameters of the distributed network in each dimension in the target short-time window can be obtained by the same method as above.
[0082] Step S303: Based on the second weight parameter of each dimension, weighted sum is performed on the load parameter of each dimension in the target short-time window to obtain the overall load degree of the distributed network in the target short-time window.
[0083] The larger the load parameter of each dimension, the greater the load of the distributed network system with respect to that dimension in the target short-time window. At the same time, the larger the second weight parameter of each dimension, the higher the confidence in the accurate analysis of the data information of that dimension. Therefore, based on the second weight parameter of each dimension, the load parameters of each dimension in the target short-time window can be weighted and summed. The overall load degree obtained can be used to reflect the overall load situation of the distributed network system in the target short-time window, providing a data basis for the subsequent calculation of the load threshold of the distributed network, thereby improving the accuracy of the calculation and analysis of the allocation adjustment coefficient of new events.
[0084] The expression of the overall load degree of the distributed network in the target short-term window can be specifically, for example, as follows:
[0085]
[0086] in, Indicates the overall load level of the distributed network in the target short-term window; Represents all nodes at all historical moments about The information entropy of the running status data in 1 dimension; Represents all nodes at all historical moments about The information entropy of the running status data in 1 dimension; Indicates the distributed network in the target short-term window about Load parameters of dimensions; Indicates the number of all dimensions of the node; Indicates The second weight parameter of the dimension.
[0087] The overall load level of the distributed network in each short-time window can be obtained by the same method as above.
[0088] Step S304: Obtain the load threshold of the distributed network according to the overall load levels of all short-term windows.
[0089] Since the distributed network system may be overloaded in certain special periods during the entire preset time period, the overall load level of all short-term windows can be analyzed to obtain the load threshold of the distributed network. If the load situation at a certain moment exceeds the load threshold, it means that the distributed network is overloaded. Subsequently, the load situation of the distributed network system at the current moment can be compared with the load threshold to determine whether the load of the distributed network system at the current moment is too large, and then the distribution adjustment coefficient of the new event can be calculated and analyzed.
[0090] Preferably, in one embodiment of the present invention, the upper quartile of the overall load level of all short-term windows is used as the load threshold of the distributed network. The calculation method of the upper quartile is a technical means well known to those skilled in the art and will not be elaborated here.
[0091] In other embodiments of the present invention, the overall load levels of all short-term windows may be input into the existing maximum inter-class variance algorithm, and the output result may be used as the load threshold of the distributed network.
[0092] Step S305: Obtain the allocation adjustment coefficient of the newly added event according to the operating status data of the target dimension of all nodes at the current moment, the second weight parameter of the target dimension, and the load threshold of the distributed network.
[0093] When the load of the distributed network system at the current moment exceeds the load threshold, it means that the overall load of the distributed network system at the current moment is too large. In the subsequent process, it is necessary to make a large adjustment to the initial similarity between the new event and the target historical event to prevent the subsequent allocation of nodes with large loads to the new event, thereby reducing the response speed of the event. On the contrary, it means that the overall load of the distributed network system at the current moment is normal, and there is no need to adjust the initial similarity. In the subsequent process, the nodes for the new events can be directly allocated based on the initial similarity. Therefore, according to the operating status data of the target dimension of all nodes at the current moment and the second weight parameter of the target dimension, the load of the distributed network system at the current moment is analyzed, and combined with the load threshold of the distributed network, the allocation adjustment coefficient of the new event is obtained.
[0094] Preferably, in one embodiment of the present invention, the method for obtaining the allocation adjustment coefficient of the newly added event further includes:
[0095] First, it is necessary to analyze the load situation of the distributed network system with respect to the target dimension at the current moment. Therefore, the overall level of the operating status data of all nodes in the target dimension at the current moment can be analyzed to obtain the load factor of the distributed network with respect to the target dimension at the current moment. The larger the load factor, the greater the load of the distributed network system with respect to the target dimension at the current moment, which provides a data basis for the subsequent calculation of the overall load index of the distributed network at the current moment.
[0096] In an embodiment of the present invention, the average value or median of the operating status data of all nodes in the target dimension at the current moment can be used as the load parameter of the distributed network with respect to the target dimension at the current moment to implement the analysis of its overall level, which is not limited here.
[0097] The same method as above can be used to obtain the load factor of each dimension of the distributed network at the current moment.
[0098] Similarly, based on the second weight parameter of each dimension, the load factors of each dimension at the current moment are weighted and summed to obtain the overall load index of the distributed network at the current moment. The overall load index reflects the overall load of the distributed network system at the current moment. The larger the overall load index, the greater the overall load of the distributed network system at the current moment. Subsequently, the overall load index of the distributed network at the current moment can be compared with the load threshold to obtain the allocation adjustment coefficient of the new event, thereby improving the effect of subsequent node allocation for the new event.
[0099] The expression of the overall load index of the distributed network at the current moment can be specifically, for example, as follows:
[0100]
[0101] in, Represents the overall load index of the distributed network at the current moment; Represents all nodes at all historical moments about The information entropy of the running status data in 1 dimension; Represents all nodes at all historical moments about The information entropy of the running status data in 1 dimension; Represents the distributed network at the current moment about The load factor of each dimension; Indicates the number of all dimensions of the node; Indicates The second weight parameter of the dimension.
[0102] From the above analysis, it can be seen that when the overall load index of the distributed network at the current moment is greater than the load threshold, and the greater the difference between the overall load index and the load threshold, the greater the overall load of the distributed network system at the current moment, then in the subsequent process, it is necessary to make a greater adjustment to the initial similarity between the new events and the target historical events, and the larger the allocation adjustment coefficient of the new events needs to be. Therefore, the difference between the overall load index of the distributed network at the current moment and the load threshold can be normalized to obtain the allocation adjustment coefficient of the new events.
[0103] In one embodiment of the present invention, the difference between the overall load index and the load threshold of the distributed network at the current moment can be calculated to analyze the difference between the two.
[0104] In one embodiment of the present invention, the normalization processing can be specifically, for example, maximum and minimum value normalization processing, and the normalization in subsequent steps can all adopt maximum and minimum value normalization processing. In other embodiments of the present invention, other normalization methods can be selected according to the specific range of numerical values, which will not be described in detail.
[0105] On the contrary, when the overall load index of the distributed network at the current moment is not greater than the load threshold, it means that the overall load of the distributed network system at the current moment is normal, and there is no need to adjust the initial similarity in subsequent steps. In the subsequent steps, the nodes of the newly added events can be directly allocated based on the initial similarity. At this time, the allocation adjustment coefficient of the newly added events can be set to 0 to facilitate subsequent calculations.
[0106] The expression of the allocation adjustment coefficient of the newly added event may be specifically, for example, as follows:
[0107]
[0108] in, represents the allocation adjustment factor for newly added events; Represents the overall load index of the distributed network at the current moment; Indicates the load threshold of the distributed network; Represents the normalization function.
[0109] Considering that the distributed network environment is in complex dynamic changes, there are large differences in the operating status of each node at different historical moments. When the overall load index of the distributed network system at the current moment exceeds the load threshold, if the initial similarity between the new event and the target historical event is adjusted only based on the allocation adjustment coefficient, the adjustment accuracy will be insufficient, resulting in a reduction in the response speed of the distributed system to the new event after the node is allocated to the new event. It is necessary to add the operating status of the node to the consideration factors of the event similarity measurement. Therefore, the embodiment of the present invention first takes the node assigned to the target historical event as the target node, and the historical moment where the target historical event is located as the target historical moment, and further analyzes the difference in the operating status data of the same dimension of the target node between the current moment and the target historical moment. The difference in the operating status obtained reflects the degree of difference in the operating status of the target node at the current moment and the target historical moment. Subsequently, based on the operating status difference and in combination with the above-obtained allocation adjustment coefficient, the initial similarity between the new event and the target historical event can be accurately adjusted.
[0110] Preferably, in one embodiment of the present invention, the method for obtaining the difference between the running status of the target node at the current moment and the target historical moment specifically includes:
[0111] According to the difference in the running status data of the target dimension between the current moment and the target historical moment of the target node, the state difference parameter of the target dimension between the current moment and the target historical moment of the target node is obtained.
[0112] In one embodiment of the present invention, the absolute value of the difference in the operating status data of the target dimension of the target node between the current moment and the target historical moment can be used as the state difference parameter of the target node at the current moment and the target historical moment regarding the target dimension to analyze the difference between the two.
[0113] The same method as above can be used to obtain the state difference parameters of the target node in each dimension at the current moment and the target historical moment.
[0114] The larger the state difference parameter of each dimension is, the greater the difference in the running state of the target node with respect to the dimension between the current moment and the target historical moment is. At the same time, the larger the second weight parameter of each dimension is, the higher the confidence in the correct analysis of the data information of each dimension is. Therefore, based on the second weight parameter of each dimension, the state difference parameter of the target node with respect to each dimension at the current moment and the target historical moment is weighted and summed to obtain the running state difference of the target node at the current moment and the target historical moment. The larger the running state difference is, the greater the difference in the running state of the target node at the current moment and the target historical moment is.
[0115] The expression of the difference between the running status of the target node at the current time and the target historical time can be specifically, for example, as follows:
[0116]
[0117] in, Indicates the difference between the running status of the target node at the current time and the target historical time; Represents all nodes at all historical moments about The information entropy of the running status data in 1 dimension; Represents all nodes at all historical moments about The information entropy of the running status data in 1 dimension; Indicates the target node's relationship between the current moment and the target historical moment State difference parameters of dimensions; Indicates the number of all dimensions of the node; Indicates The second weight parameter of the dimension.
[0118] At this point, the allocation adjustment coefficient of the newly added event and the difference in the operating status of the target node at the current time and the target historical time are obtained.
[0119] Step S4: According to the allocation adjustment coefficient and the difference in the running status, the initial similarity is corrected to obtain the real similarity between the new event and the target historical event; based on the real similarity between the new event and each historical event in each node, the new event is allocated to a node.
[0120] In the subsequent steps of the embodiment of the present invention, it is necessary to assign the new events to the nodes corresponding to the historical events with higher similarity for processing. Therefore, in order to avoid load imbalance of the distributed network system after assigning nodes to the new events and improve the response speed of the distributed network system to the new events after assigning nodes to the new events, it is necessary to correct the initial similarity based on the allocation adjustment coefficient and operating status difference obtained above, and accurately reflect the similarity between the new events and the historical events through the acquired real similarity, thereby improving the effect of subsequent node allocation for the new events.
[0121] Preferably, in one embodiment of the present invention, the method for obtaining the real similarity between the newly added event and the target historical event specifically includes:
[0122] The smaller the difference between the running states of the target node at the current moment and the target historical moment, the greater the similarity of the running states of the target node at the current moment and the target historical moment. Therefore, the running state difference can be normalized with negative correlation to obtain the running state similarity of the target node at the current moment and the target historical moment. Subsequently, the running state similarity can be involved in the calculation process of the real similarity, so as to realize the running state of the node as an influencing factor of the similarity measurement. The greater the running state similarity, the greater the adjustment degree of the initial similarity.
[0123] From the above analysis, it can be seen that when the allocation adjustment coefficient of the newly added event is not equal to 0, the initial similarity needs to be adjusted to a greater extent. When the allocation adjustment coefficient of the newly added event is equal to 0, the initial similarity can be directly used as the similarity measure between the newly added event and the target historical event. Therefore, the product value of the allocation adjustment coefficient and the operating state similarity can be used as the adjustment parameter. The larger the adjustment parameter, the greater the degree of adjustment of the initial similarity. Then, the product value of the initial similarity and the adjustment parameter can be used as the adjustment amount, and the sum of the initial similarity and the adjustment amount can be used as the true similarity between the newly added event and the target historical event.
[0124] The expression of the real similarity between the newly added event and the target historical event can be specifically, for example, as follows:
[0125]
[0126] in, Indicates the true similarity between the newly added event and the target historical event; Indicates the initial similarity between the newly added event and the target historical event; represents the allocation adjustment factor for newly added events; Indicates the difference between the running status of the target node at the current time and the target historical time; Represented by natural function An exponential function with base .
[0127] The same method as above can be used to obtain the true similarity between the new event and each historical event, and then the true similarity is used to measure the similarity between the new event and each historical event. Since the purpose of the embodiment of the present invention is to assign the new event to the node corresponding to the historical event with higher similarity, the new event can be assigned to the node based on the true similarity between the new event and each historical event in each node, thereby keeping the load of the distributed network system balanced and improving the speed of processing and responding to the new event.
[0128] Preferably, in one embodiment of the present invention, the method for assigning nodes to newly added events specifically includes:
[0129] See also Figure 3 , which shows a flow chart of a method for node allocation for newly added events provided by an embodiment of the present invention.
[0130] Step S401: Obtain the nodes to be allocated and the priority allocation degree of each node to be allocated according to the real similarity between the newly added event and each historical event in each node.
[0131] The greater the real similarity between a new event and a historical event, the more necessary it is to assign the new event to the node corresponding to the historical event for processing. Therefore, the real similarity between the new event and each historical event in each node can be analyzed, and the nodes to be assigned and the priority allocation degree of each node to be assigned can be selected from all nodes. The greater the priority allocation degree of each node to be assigned, the more necessary it is to assign the new event to the node to be assigned. Subsequently, the new event is assigned to a node based on the priority allocation degree of each node to be assigned.
[0132] Preferably, in one embodiment of the present invention, the method for acquiring the nodes to be allocated and the priority allocation degree of each node to be allocated specifically includes:
[0133] A preset number of historical events corresponding to the largest true similarities are selected as similar historical events. The similarity between similar historical events and newly added events is relatively large. The nodes where each similar historical event is located are used as nodes to be allocated. Some nodes to be allocated may contain multiple similar historical events. Therefore, the overall level of the true similarity between the newly added event and all similar historical events in each node to be allocated can be analyzed to obtain the priority allocation degree of each node to be allocated. The preset number is set to 30, and the specific value of the preset number can also be set by the implementer according to the specific implementation scenario, which is not limited here.
[0134] In an embodiment of the present invention, the average or median of the real similarity between the new event and all similar historical events in each node to be assigned can be used as the priority allocation degree of each node to be assigned, so as to realize the analysis of the overall level of the real similarity between the new event and all similar historical events in each node to be assigned, which is not limited here.
[0135] Step S402: performing node allocation for the newly added event based on the priority allocation degree of each node to be allocated.
[0136] The greater the priority of each node to be assigned, the more priority the newly added event needs to be assigned to the node. Therefore, based on the priority of each node to be assigned, the newly added event can be assigned to a node, thereby improving the effect of node assignment for the newly added event.
[0137] Preferably, in an embodiment of the present invention, the newly added event may be allocated to the to-be-allocated node corresponding to the maximum priority allocation degree.
[0138] In other embodiments of the present invention, the load condition of the node to be allocated at the current moment may also be considered in the allocation process. For example, when the CPU usage or network bandwidth usage of the node to be allocated corresponding to the maximum priority allocation degree exceeds a preset threshold, the node to be allocated corresponding to the second largest priority allocation degree may be selected, and the load condition of the node may be analyzed to see whether it exceeds the preset threshold until the new event is successfully allocated to the node to be allocated, wherein the preset threshold is set to 0.7, and the specific value of the preset threshold may also be set by the implementer according to the specific implementation scenario, which is not limited here.
[0139] In summary, the embodiment of the present invention first obtains the newly added events of the unassigned nodes in the distributed network central system at the current moment, and the historical events of the assigned nodes at each historical moment in a preset time period, and at the same time obtains the operating status data of each node in different dimensions at the current moment and each historical moment, and the newly added events and historical events both include basic event data of different fields; any historical event is taken as the target historical event, and the initial similarity between the newly added event and the target historical event is obtained according to the difference in the basic event data of the same field between the newly added event and the target historical event; the preset time period is divided into multiple short-term windows, and the operation status data of each node in each dimension at each historical moment in the short-term window is obtained according to the difference in the basic event data of each node at each historical moment in the short-term window. The operation status data of the target historical event and the operation status data of all nodes in each dimension at the current moment are used to obtain the allocation adjustment coefficient of the new event; the node allocated to the target historical event is taken as the target node, and the historical moment of the target historical event is taken as the target historical moment. According to the difference in the operation status data of the target node in the same dimension between the current moment and the target historical moment, the operation status difference of the target node at the current moment and the target historical moment is obtained; according to the allocation adjustment coefficient and the operation status difference, the initial similarity is corrected to obtain the real similarity between the new event and the target historical event; based on the real similarity between the new event and each historical event in each node, the new event is allocated a node.
[0140] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0141] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A distributed event hub networking method, characterized in that: The method comprises: Obtain new events of unassigned nodes in the distributed network central system at the current moment, and historical events of assigned nodes at each historical moment in a preset time period, and at the same time obtain the operating status data of each node in different dimensions at the current moment and each historical moment, wherein both new events and historical events include basic event data of different fields; Taking any historical event as a target historical event, obtaining the initial similarity between the newly added event and the target historical event according to the difference of the event basic data of the same field between the newly added event and the target historical event; The preset time period is divided into multiple short-term windows, and the allocation adjustment coefficient of the new event is obtained according to the operating status data of all nodes in each dimension at each historical moment in the short-term window, and the operating status data of all nodes in each dimension at the current moment; the node assigned to the target historical event is taken as the target node, and the historical moment where the target historical event is located is taken as the target historical moment, and the operating status difference of the target node at the current moment and the target historical moment is obtained according to the difference of the operating status data of the target node in the same dimension between the current moment and the target historical moment; According to the allocation adjustment coefficient and the running state difference, the initial similarity is corrected to obtain the real similarity between the new event and the target historical event; based on the real similarity between the new event and each historical event in each node, the new event is allocated to a node.
2. A distributed event hub networking method according to claim 1, characterized in that: The obtaining the initial similarity between the newly added event and the target historical event according to the difference of the event basic data of the same field between the newly added event and the target historical event includes: Taking any field as the target field, normalizing the information entropy of the event basic data of the newly added event and all historical events in the target field to obtain a first weight parameter of the target field, wherein the sum of the first weight parameters of all fields is equal to a value of 1; Performing negative correlation normalization processing on the difference of the event basic data between the newly added event and the target historical event in the target field to obtain the data similarity between the newly added event and the target historical event with respect to the target field; Based on the first weight parameter of each field, a weighted sum is performed on the data similarity between the newly added event and the target historical event in each field to obtain an initial similarity between the newly added event and the target historical event.
3. A distributed event hub networking method according to claim 1, characterized in that: The obtaining of the allocation adjustment coefficient of the newly added event according to the operation status data of each dimension of all nodes at each historical moment in the short-time window and the operation status data of each dimension of all nodes at the current moment includes: Taking any dimension of the node as the target dimension, normalizing the information entropy of the operating status data of the target dimension of all nodes at all historical moments to obtain a second weight parameter of the target dimension, wherein the sum of the second weight parameters of all dimensions of the node is equal to a value of 1; Taking any short-time window as a target short-time window, analyzing the overall level of the operating status data of the target dimension of all nodes at all historical moments in the target short-time window, and obtaining the load parameter of the distributed network with respect to the target dimension in the target short-time window; Based on the second weight parameter of each dimension, weighted sum is performed on the load parameter of each dimension in the target short-time window to obtain the overall load degree of the distributed network in the target short-time window; Obtaining a load threshold of the distributed network according to the overall load levels of all short-term windows; The distribution adjustment coefficient of the new event is obtained based on the operating status data of the target dimension of all nodes at the current moment, the second weight parameter of the target dimension, and the load threshold of the distributed network.
4. A distributed event hub networking method according to claim 3, characterized in that: The step of obtaining a load threshold of the distributed network according to the overall load level of all short-term windows includes: The upper quartile of the overall load level of all short-term windows is used as the load threshold of the distributed network.
5. A distributed event hub networking method according to claim 3, characterized in that: The obtaining of the distribution adjustment coefficient of the newly added event according to the operating status data of the target dimension of all nodes at the current moment, the second weight parameter of the target dimension, and the load threshold of the distributed network includes: Analyze the overall level of the operating status data of all nodes in the target dimension at the current moment to obtain the load factor of the distributed network with respect to the target dimension at the current moment; Based on the second weight parameter of each dimension, weighted summation is performed on the load factor of each dimension at the current moment to obtain the overall load index of the distributed network at the current moment; When the overall load index of the distributed network at the current moment is greater than the load threshold, normalizing the difference between the overall load index of the distributed network at the current moment and the load threshold to obtain a distribution adjustment coefficient of the newly added event; When the overall load index of the distributed network at the current moment is not greater than the load threshold, the allocation adjustment coefficient of the newly added event is set to 0.
6. A distributed event hub networking method according to claim 3, characterized in that: The obtaining the difference in the running state of the target node at the current moment and the target historical moment according to the difference in the running state data of the same dimension between the current moment and the target historical moment comprises: According to the difference of the running state data of the target dimension between the current moment and the target historical moment of the target node, a state difference parameter of the target dimension between the current moment and the target historical moment of the target node is obtained; Based on the second weight parameter of each dimension, a weighted sum is performed on the state difference parameter of the target node at the current moment and the target historical moment with respect to each dimension to obtain the running state difference of the target node at the current moment and the target historical moment.
7. A distributed event hub networking method according to claim 1, characterized in that: The step of correcting the initial similarity according to the allocation adjustment coefficient and the running state difference to obtain the real similarity between the newly added event and the target historical event includes: Performing negative correlation normalization processing on the running state difference to obtain the running state similarity of the target node at the current moment and the target historical moment; The product value of the allocation adjustment coefficient and the operating state similarity is used as an adjustment parameter; the product value of the initial similarity and the adjustment parameter is used as an adjustment amount; The sum of the initial similarity and the adjustment amount is used as the true similarity between the newly added event and the target historical event.
8. A distributed event hub networking method according to claim 1, characterized in that: The node allocation for the new event based on the real similarity between the new event and each historical event in each node includes: According to the true similarity between the newly added event and each historical event in each node, the nodes to be allocated and the priority allocation degree of each node to be allocated are obtained; Based on the priority allocation degree of each node to be allocated, node allocation is performed on the newly added event.
9. A distributed event hub networking method according to claim 8, characterized in that: The obtaining of the nodes to be allocated and the priority allocation degree of each node to be allocated according to the real similarity between the newly added event and each historical event in each node includes: Selecting a preset number of historical events corresponding to the largest true similarities as similar historical events, and taking the nodes where the similar historical events are located as nodes to be allocated; The overall level of the true similarity between the newly added event and all similar historical events in each node to be assigned is analyzed to obtain the priority assignment degree of each node to be assigned.
10. A distributed event hub networking method according to claim 8, characterized in that: The node allocation for the newly added event based on the priority allocation degree of each node to be allocated includes: Assign the newly added events to the to-be-assigned node corresponding to the maximum priority assignment degree.
