Data synchronization method for customer demand management system and related equipment

Abstract

The invention provides a data synchronization method for a customer demand management system and related equipment, and the method comprises the steps: determining the competition margin of a receiving node through configuration information and a distribution log, auditing demand data synchronously received by the receiving node, obtaining the auditing amount during the synchronization of the demand data, and carrying out the verification of the demand data. Determining a synchronous scheduling sequence of the receiving node during synchronous receiving of the demand data according to the auditing amount and the competition margin; determining a synchronous update state of the receiving node through the receiving node index record, and performing state authentication on a synchronization process of the demand data according to the synchronous update state to obtain a synchronous state entropy; and performing synchronous optimization on the configuration information of the receiving node according to the synchronous scheduling sequence and the synchronous state entropy to obtain an optimized configuration curve, and adjusting the synchronization priority of the receiving node according to the configuration information on the optimized configuration curve. By adopting the scheme provided by the invention, the optimal scheduling of the synchronization priority of the receiving nodes can be realized, so that the auditing efficiency of a customer demand management system during data synchronization is improved.

Description

Technical Field

[0001] This application relates to the technical field of data synchronization. More specifically, this application relates to a data synchronization method and related devices for a customer demand management system. Background Art

[0002] Data synchronization refers to the process of ensuring that multiple nodes receive the same data information at the same time in a distributed system. Data synchronization has a wide range of application scenarios, including: database synchronization, file synchronization, application data synchronization, and other data synchronizations. Data synchronization usually involves data replication, transmission, and consistency verification, and is an important technology to ensure system stability and data accuracy.

[0003] Data synchronization in a customer demand management system refers to the process of ensuring the consistency and real-time update of customer demand information among different users in the customer demand management system. Existing customer demand management systems usually rely on a distributed architecture with multiple nodes. However, in the case of concurrent multi-node resources in the customer demand management system, there is a large synchronization competition among receiving nodes when requesting data updates, and it is difficult to balance the allocation and scheduling of configured resources among nodes, resulting in data deviation and reception delay when different receiving nodes synchronously receive demand data. Therefore, how to achieve optimized scheduling of receiving nodes to improve the auditing efficiency of the customer demand management system during data synchronization has become a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides a data synchronization method and related devices for a customer demand management system, which can achieve optimized scheduling of the synchronization priority of receiving nodes, thereby improving the auditing efficiency of the customer demand management system during data synchronization.

[0005] In a first aspect, this application provides a data synchronization method for a customer demand management system, including the following steps:

[0006] Obtain the synchronization record of demand data to obtain demand synchronization data;

[0007] Determine the competition margin of each receiving node when synchronously receiving demand data according to the configuration information of each receiving node during the synchronization process of the demand data and the allocation log in the demand synchronization data;

[0008] Audit the demand data synchronously received by all receiving nodes to obtain the audit amount during demand data synchronization, and then determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data based on the audit amount and each competition margin;

[0009] Determine the synchronization update status of each receiving node based on the index records of the demand data for each receiving node, and then perform status authentication on the synchronization process of the demand data through each synchronization update status to obtain the synchronization state entropy of all receiving nodes;

[0010] Synchronously optimize the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve meets the smooth standard, adjust the synchronization priorities of each receiving node according to the configuration information on the optimized configuration curve.

[0011] In some embodiments, determining the competition margin of each receiving node when synchronously receiving demand data according to the configuration information of each receiving node during the synchronization process of the demand data and the allocation log in the demand synchronization data specifically includes:

[0012] Obtain the configuration information of each receiving node during the synchronization process of the demand data;

[0013] Determine the delay characteristics of each receiving node during the synchronization process of the demand data according to the configuration information of each receiving node;

[0014] Extract the allocation logs of all receiving nodes during the synchronization of the demand data from the demand synchronization data;

[0015] Determine the task allocation list of all receiving nodes during the synchronization of the demand data through the allocation log;

[0016] Determine the competition margin of each receiving node when synchronously receiving demand data from the task allocation list and the delay characteristics of each receiving node during the synchronization process of the demand data.

[0017] In some embodiments, auditing the demand data synchronously received by all receiving nodes to obtain the audit volume during the synchronization of the demand data specifically includes:

[0018] Determine the attribute entropy of each receiving node;

[0019] Obtain the configuration information of each receiving node, and then determine the configuration score of each receiving node according to the configuration information of each receiving node;

[0020] Determine the audit coefficient of each receiving node through the configuration score and the attribute entropy of each receiving node;

[0021] Obtain the data information flow of each receiving node synchronously receiving the demand data;

[0022] Perform information auditing on the data information flow of each receiving node based on the audit coefficient of each receiving node, and then obtain the audit volume during the synchronization of the demand data.

[0023] In some embodiments, the synchronization scheduling sequence of all receiving nodes when determining the synchronous reception demand data from the audit quantity and each competition margin specifically includes:

[0024] Determine the coordination degree of each receiving node when determining the synchronous reception demand data according to the audit quantity;

[0025] Cooperatively schedule the process of synchronously receiving demand data through the coordination degree and competition margin of each receiving node to obtain the synchronization scheduling value of each receiving node;

[0026] Based on the synchronization scheduling value of each receiving node, determine the synchronization scheduling sequence of all receiving nodes when determining the synchronous reception demand data.

[0027] In some embodiments, determining the synchronous update status of each receiving node based on the index record of the demand data of each receiving node specifically includes:

[0028] Obtain the index record of the demand data of each receiving node;

[0029] Extract the synchronization delay of each receiving node from each index record;

[0030] Obtain the data information flow of each receiving node for synchronously receiving demand data;

[0031] Determine the synchronous update rate of each receiving node through the synchronization delay and data information flow of each receiving node, and further determine the synchronous update status of each receiving node.

[0032] In some embodiments, synchronously optimizing the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes specifically includes:

[0033] Obtain the conditional entropy of each receiving node when synchronously receiving demand data;

[0034] Determine the optimization order of each receiving node according to the conditional entropy of each receiving node and the synchronization state entropy;

[0035] Optimize the configuration information of all receiving nodes through the optimization order, the synchronization state entropy, and each synchronization scheduling value in the synchronization scheduling sequence, and monitor the configuration status of all receiving nodes to obtain the optimized configuration curve of all receiving nodes.

[0036] In some embodiments, the allocation log is a database log.

[0037] In a second aspect, the present application provides a customer demand management system, and the customer demand management system includes a data synchronization unit, and the data synchronization unit includes:

[0038] An acquisition module, configured to acquire a synchronization record of requirement data to obtain requirement synchronization data;

[0039] A processing module, configured to determine a competition margin when each receiving node synchronously receives requirement data according to the configuration information of each receiving node during the synchronization process of the requirement data and the allocation log in the requirement synchronization data;

[0040] The processing module is further configured to audit the requirement data synchronously received by all receiving nodes to obtain an audit volume during requirement data synchronization, and further determine a synchronization scheduling sequence of all receiving nodes when synchronously receiving requirement data according to the audit volume and each competition margin;

[0041] The processing module is further configured to determine a synchronization update status of each receiving node based on an index record of the requirement data of each receiving node, and further perform status authentication on the synchronization process of the requirement data through each synchronization update status to obtain a synchronization state entropy of all receiving nodes;

[0042] An execution module, configured to synchronously optimize the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain an optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches a smoothing standard, adjust the synchronization priorities of each receiving node according to the configuration information on the optimized configuration curve.

[0043] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to acquire the code and execute the above data synchronization method for a customer requirement management system.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above data synchronization method for a customer requirement management system is implemented.

[0045] The technical solution provided by the embodiments disclosed in the present application has the following beneficial effects:

[0046] In the data synchronization method and related devices provided by this application for a customer demand management system, first, obtain the synchronization record of demand data to obtain demand synchronization data; second, determine the competition margin of each receiving node when synchronously receiving demand data according to the configuration information of each receiving node during the synchronization process of the demand data and the allocation log in the demand synchronization data; then, audit the demand data synchronously received by all receiving nodes to obtain the audit volume during demand data synchronization, and further determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data from the audit volume and each competition margin; then, determine the synchronization update status of each receiving node based on the index record of the demand data of each receiving node, and further perform status authentication on the synchronization process of the demand data through each synchronization update status to obtain the synchronization state entropy of all receiving nodes; finally, synchronously optimize the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, adjust the synchronization priority of each receiving node according to the configuration information on the optimized configuration curve.

[0047] It can be seen that this application adjusts the synchronization priority of all receiving nodes through the configuration information on the optimized configuration curve to complete the synchronization optimization of demand data. First, a synchronization scheduling sequence is composed of the synchronization scheduling values of each receiving node. Among them, the synchronization scheduling value can be used to measure the scheduling priority of the receiving node when synchronously receiving demand data, and the demand data received by different receiving nodes is audited based on the synchronization scheduling value, which is beneficial to determining the scheduling values and optimization order of all receiving nodes during the demand data synchronization process, and then optimizing the receiving nodes through the determined scheduling values and optimization order to improve the accuracy of demand data during synchronization. Second, by determining the synchronization state entropy, the synchronization conditions of all receiving nodes when synchronously receiving demand data are clarified, which is beneficial to identifying unstable receiving nodes during the demand data synchronization process, and the conditions of unstable receiving nodes in the system can be configured according to the synchronization state entropy, thereby optimizing the concurrent efficiency of all receiving nodes. Finally, synchronously optimize the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve, and adjust the synchronization priority of the receiving nodes according to the configuration information on the optimized configuration curve, which is convenient for reducing the resource occupancy rate during demand data synchronization and is beneficial to the high-concurrency synchronous reception of demand data by receiving nodes. In summary, the solution of this application can achieve the optimized scheduling of the synchronization priority of receiving nodes, thereby improving the audit efficiency of the customer demand management system during data synchronization. Brief Description of the Drawings

[0048] Figure 1 is an exemplary flowchart of a data synchronization method for a customer demand management system shown according to some embodiments of this application;

[0049] Figure 2 is a schematic flowchart of determining a competition margin as shown in some embodiments of the present application;

[0050] Figure 3 is a schematic flowchart of determining a synchronization state entropy as shown in some embodiments of the present application;

[0051] Figure 4 is a schematic structural diagram of a data synchronization unit in some embodiments of the present application;

[0052] Figure 5 is a schematic structural diagram of a computer device for implementing a data synchronization method for a customer demand management system as shown in some embodiments of the present application. Detailed implementation manners

[0053] To better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0054] Refer to Figure 1 , which is an exemplary flowchart of a data synchronization method for a customer demand management system as shown in some embodiments of the present application. The data synchronization method 100 for the customer demand management system mainly includes the following steps:

[0055] In step 101, obtain the synchronization record of the demand data to obtain the demand synchronization data.

[0056] Specifically, when implemented, obtaining the synchronization record of the demand data to obtain the demand synchronization data can be implemented by the following method, that is: obtain the synchronization record of the target customer demand data within a specified time period from the database of the customer demand management system, and use the set of all synchronization records as the demand synchronization data of the target customer. Wherein, the specified time is the time period from the current moment to the past week, and the synchronization record includes: creation time, creator, demand title, demand description, priority of the demand, demand processing status, and allocation log. In addition, the allocation log is a database log, and the allocation log represents the set of information at each receiving node when the demand data in the customer demand management system is synchronized.

[0057] It should be noted that the database of the customer demand management system in the present application is a distributed database composed of two or more data nodes. The data nodes include a master node and receiving nodes. Among them, the master node is the creation node of the synchronization record. Each receiving node corresponds to a different user of the customer demand management system when the demand data is synchronized. The different users include: customers, customer service, R & D teams, data analysts, and other system users.

[0058] In step 102, the competition margin of each receiving node when synchronously receiving demand data is determined according to the configuration information of each receiving node during the synchronization process in the demand data and the allocation log in the demand synchronization data.

[0059] In some embodiments, as Figure 2 shown, the figure is a schematic flowchart of determining the competition margin shown in some embodiments of the present application. In this embodiment, the competition margin of each receiving node when synchronously receiving demand data can be implemented according to the configuration information of each receiving node during the synchronization process in the demand data and the allocation log in the demand synchronization data, which can be achieved by the following steps:

[0060] In step 1021, obtain the configuration information of each receiving node during the synchronization process of the demand data;

[0061] In step 1022, determine the latency characteristics of each receiving node during the synchronization process of the demand data according to the configuration information of each receiving node;

[0062] In step 1023, extract the allocation logs of all receiving nodes when the demand data is synchronized from the demand synchronization data;

[0063] In step 1024, determine the task allocation list of all receiving nodes when the demand data is synchronized through the allocation log;

[0064] In step 1025, determine the competition margin of each receiving node when synchronously receiving demand data from the task allocation list and the latency characteristics of each receiving node during the synchronization process of the demand data.

[0065] Specifically, obtaining the configuration information of each receiving node during the synchronization process of the demand data can be implemented in the following way, that is: when the demand data is synchronized, load the interface configuration file from the customer demand management system, and obtain the configuration information of each receiving node from the interface configuration file. Among them, the suffix of the interface configuration file can be ".yaml", and the configuration information includes the configuration instructions, hardware configuration, network configuration and processing capacity of the corresponding node; determining the latency characteristics of each receiving node during the synchronization process of the demand data according to the configuration information of each receiving node can be implemented in the following way, that is: the processing capacity of the receiving node includes: processing time, data size and disk load. For each receiving node, the latency in the disk load of the receiving node can be used as the latency characteristic of the receiving node during the synchronization process of the demand data, and then the latency characteristics of all receiving nodes during the synchronization process of the demand data can be obtained. Among them, the disk load of the receiving node includes: IOPS (Input / Output Operations Per Second), throughput and latency.

[0066] In addition, in specific implementation, the allocation logs of all receiving nodes during the synchronization of requirement data can be extracted from the synchronized requirement data in the following manner, that is: the allocation log refers to the set of information at each receiving node during the synchronization of requirement data, and the information at each receiving node includes: node identifier, task identifier, task allocation time, task status, synchronization time, and node configuration parameters. A database query statement can be written to extract the information at each receiving node through the database query statement, and the set of all information is used as the allocation log of all receiving nodes during the synchronization of requirement data.

[0067] In specific implementation, the task allocation list of all receiving nodes during the synchronization of requirement data can be determined through the allocation log in the following manner, that is: the list composed of the proportion of the task allocation time of each receiving node (i.e., the ratio of the task allocation time of each receiving node to the sum of all task allocation times) in ascending order is used as the task allocation list of all receiving nodes during the synchronization of requirement data; the competitive margin of each receiving node when synchronously receiving requirement data can be determined from the task allocation list and the delay characteristics of each receiving node during the synchronization of requirement data in the following manner, that is: the task allocation list includes the proportion of the task allocation time of each receiving node, and the product of each proportion of the task allocation time and the corresponding delay characteristic can be used as the competitive margin of the corresponding receiving node when synchronously receiving requirement data.

[0068] It should be noted that the competitive margin in this application is an indicator to measure the task processing ability of the receiving node during the synchronization of requirement data. The competitive margin reflects the elastic space of each receiving node during the data synchronization process for processing multiple tasks simultaneously, that is: the receiving node with a higher competitive margin has a stronger ability to handle multi-task concurrency, and the receiving node with a lower competitive margin has limited ability to synchronously process multiple tasks, which is likely to lead to a longer delay.

[0069] In step 103, the requirement data synchronously received by all receiving nodes is audited to obtain the audit volume during the synchronization of requirement data, and then the synchronous scheduling sequence of all receiving nodes when synchronously receiving requirement data is determined from the audit volume and each competitive margin.

[0070] In some embodiments, the audit volume during the synchronization of requirement data can be obtained by auditing the requirement data synchronously received by all receiving nodes through the following steps:

[0071] Determine the attribute entropy of each receiving node;

[0072] Obtain the configuration information of each receiving node, and then determine the configuration score of each receiving node according to the configuration information of each receiving node;

[0073] Determine the audit coefficient of each receiving node based on the configuration score and attribute entropy of each receiving node;

[0074] Obtain the data information flow of each receiving node for synchronously receiving demand data;

[0075] Perform information auditing on the data information flow of each receiving node based on the audit coefficient of each receiving node, and then obtain the audit amount during demand data synchronization.

[0076] When specifically implemented, the attribute entropy of each receiving node can be determined in the following way: that is, the attribute entropy is an index to measure the stability degree of the state attributes of the receiving node. The higher the attribute entropy of the receiving node, the more complex the state attributes of the receiving node and the greater the uncertainty of the receiving node state. The lower the attribute entropy of the receiving node, the more stable the state attributes of the receiving node and the greater the predictability of the receiving node state. The state attributes include: processing capacity, network latency, disk I / O performance, data processing time, and load balancing. For each receiving node, the information entropy of each attribute of the receiving node can be described respectively, and then the sum of the information entropies of each attribute is used as the attribute entropy of the receiving node, and then the attribute entropies of all receiving nodes are obtained.

[0077] In addition, when specifically implemented, the configuration information of each receiving node is obtained, and then the configuration score of each receiving node can be determined according to the configuration information of each receiving node in the following way: that is, the configuration information of each receiving node includes the processing time, data size, and disk load of the corresponding receiving node. For each receiving node, the processing time, data size, and disk load of the receiving node are standardized using standard scores in the prior art, and the weighted sum value of the standardized processing time, data size, and disk load is used as the configuration score of the receiving node, and then the configuration scores of all receiving nodes are obtained. Among them, for the weighted sum of the standardized processing time, data size, and disk load, the weight value of the weighted sum can be adaptively set according to the actual ratio of the standardized processing time, data size, and disk load of each interface.

[0078] When specifically implemented, the audit coefficient of each receiving node can be determined based on the configuration score and attribute entropy of each receiving node in the following way: that is, the audit coefficient is a parameter to measure the audit ability of the receiving node. The configuration scores and attribute entropies of all receiving nodes can be input into an initialized regression analysis model for model training. For each receiving node, the configuration score and attribute entropy of this receiving node are respectively input into this regression analysis model for prediction, and then the value of the linear combination output by this regression analysis model is used as the audit coefficient of this receiving node, and then the audit coefficients of each receiving node are obtained.

[0079] In addition, the data information flow for obtaining the data of the synchronization reception requirements of each receiving node can be implemented in the following manner, that is: the Wireshark network analysis tool can be used to capture the data flow at each receiving node during the synchronization of the requirement data, and the set composed of all the data flows captured at each receiving node is used as the data information flow for the corresponding receiving node to synchronize and receive the requirement data. In other embodiments, other network analysis tools can also be used for capture, which is not limited here.

[0080] When specifically implemented, information auditing is performed on the data information flow of each receiving node based on the auditing coefficient of each receiving node, and then the auditing amount during the synchronization of the requirement data can be implemented in the following manner, that is: First, the hash values of the data information flow of each node are respectively extracted using the hash function in the prior art, and then all the receiving nodes are sorted by hash using all the hash values; Then, for each receiving node, the ratio of the receiving node priority obtained after sorting the receiving node by hash is used as the weight value of the receiving node, the product of the weight value of the receiving node and the hash value is used as the weighted hash value of the receiving node, and the product of the weighted hash value of the receiving node and the amount of information of the data information flow is used as the auditing value of the receiving node, thereby obtaining the auditing value of each receiving node; Finally, the standard deviation of all the auditing values is used as the auditing amount during the synchronization of the requirement data.

[0081] It should be noted that the auditing amount in this application is an index for measuring the degree of unity of the requirement information synchronously received by different receiving nodes, that is: the larger the auditing amount, the more unified the requirement information synchronously received by different receiving nodes, the smaller the auditing amount, the greater the deviation of the requirement information synchronously received by different receiving nodes. The auditing amount can also represent the data integrity of the requirement data received by all receiving interfaces. A higher auditing amount indicates that the data reception status of each receiving node is relatively stable and the data synchronization quality is high, while a lower auditing amount indicates that there is an abnormality in the data reception of the receiving node. Abnormal screening can be performed according to the auditing values of each interface, and the screened abnormal receiving nodes can be optimized and scheduled, which is beneficial to improving the consistency of the requirement data received by the receiving nodes.

[0082] In some embodiments, the synchronization scheduling sequence of all receiving nodes when synchronously receiving the requirement data can be determined from the auditing amount and each competition margin by the following steps:

[0083] Determine the coordination degree of each receiving node when synchronously receiving the requirement data according to the auditing amount;

[0084] Cooperatively schedule the process of synchronously receiving the requirement data through the coordination degree and competition margin of each receiving node to obtain the synchronization scheduling value of each receiving node;

[0085] Determine the synchronization scheduling sequence of all receiving nodes when determining the synchronization reception demand data based on the synchronization scheduling values of each receiving node.

[0086] In specific implementation, when determining the synchronization reception demand data according to the audit quantity, the coordination degree of each receiving node can be implemented in the following manner, that is: for each receiving node, take the absolute difference between the audit value of the receiving node and the audit quantity as the audit difference of the receiving node, and then take the audit quantity as the normalization standard to normalize the audit difference of the receiving node to obtain the normalized audit difference (i.e., audit difference / audit quantity). As a preferred embodiment, the coordination degree of the receiving node when synchronously receiving demand data can be implemented using the following formula, that is: coordination degree = 1 / (1 + normalized audit difference), and then the coordination degree of each receiving node when synchronously receiving demand data can be obtained.

[0087] In addition, in specific implementation, the process of synchronously receiving demand data can be cooperatively scheduled through the coordination degree and competition margin of each receiving node, and the synchronization scheduling value of each receiving node can be obtained in the following manner, that is: use the maximum-minimum normalization method in the prior art to normalize the coordination degree and competition margin of all receiving nodes respectively, and obtain the time delay of all receiving nodes when synchronously receiving demand data, and then plot the time delay in a statistical graph. For each receiving node, perform a weighted sum of the coordination degree and competition margin of the receiving node. At this time, the weight values of the weighted sum are each set to 0.5, and the weight values of the coordination degree and competition margin are adjusted (increased and decreased) respectively, and ensure that the sum of the weight values of the coordination degree and competition margin is 1. When the time delay in the statistical graph tends to be the smallest and stable, take the value obtained by performing a weighted sum of the coordination degree and competition margin of the receiving node as the synchronization scheduling value of the receiving node, and then the synchronization scheduling value of each receiving node can be obtained; determining the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data based on the synchronization scheduling value of each receiving node can be implemented in the following manner, that is: take the sequence randomly composed of all synchronization scheduling values as the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data.

[0088] It should be noted that the synchronization scheduling sequence in this application refers to the sequence composed of the synchronization scheduling values of each receiving node. Among them, the synchronization scheduling value is used to measure the scheduling priority of the receiving node when synchronously receiving demand data. The higher the synchronization scheduling value, the more preferentially the receiving node is processed during the synchronization process, and the greater the competition strength of the receiving node. The lower the synchronization scheduling value, the more late the receiving node is processed during the synchronization process, and the smaller the competition strength of the receiving node. Through the synchronization scheduling sequence, it can be ensured that during the synchronization process of demand data, each receiving node can be processed in the optimal order, which can improve the consistency of demand data during synchronization.

[0089] In step 104, based on the index records of the demand data for each receiving node, determine the synchronous update status of each receiving node, and then perform status authentication on the synchronization process of the demand data through each synchronous update status to obtain the synchronization state entropy of all receiving nodes.

[0090] In some embodiments, determining the synchronous update status of each receiving node based on the index records of the demand data for each receiving node can be implemented by the following steps:

[0091] Obtain the index records of the demand data for each receiving node;

[0092] Extract the synchronization delay of each receiving node from each index record;

[0093] Obtain the data information flow of each receiving node for synchronously receiving the demand data;

[0094] Determine the synchronous update rate of each receiving node through the synchronization delay and data information flow of each receiving node, and then determine the synchronous update status of each receiving node.

[0095] Specifically, obtaining the index records of the demand data for each receiving node can be implemented in the following way, that is: obtain the operation logs of each receiving node, and extract the index records of the demand data for the corresponding receiving node from each operation log. The operation log refers to the log file generated by each receiving node during the process of processing the demand data. The operation log records the reception, processing, and storage of the demand data by the receiving node. The operation log includes: timestamp, node identifier, operation type, processing duration, data size, and other log information. The index record is the specific information extracted from the operation log and is used to track how each receiving node processes the demand data. The index record includes the node identifier, operation time, and data size. The index record is helpful for analyzing the efficiency of demand data synchronization.

[0096] In addition, extracting the synchronization delay of each receiving node from each index record can be implemented in the following way, that is: first, extract the timestamp information from the index records of each receiving node respectively. For each receiving node, obtain the start time and end time of the demand data synchronization from the timestamp information of the receiving node, and use the end time and start time as the synchronization delay of the receiving node, and then obtain the synchronization delays of all receiving nodes. The timestamp information refers to the specific time marked for each operation in the operation log. The timestamp information can record the time sequence of events. The synchronization delay refers to the time spent from the start of processing the demand data at the receiving node to the end of processing, and is used to measure the efficiency of the receiving node in processing the demand data.

[0097] In specific implementation, the synchronization update rate of each receiving node is determined based on the synchronization delay and data information flow of each receiving node, and then the synchronization update status of each receiving node can be determined in the following manner: First, the size of the data packet in the data information flow of each receiving node is used as the data packet size of each receiving node. For each receiving node, the synchronization update rate of this receiving node can be calculated using the following formula: Synchronization update rate = data packet size * synchronization delay, and then the synchronization update rate of each receiving node is obtained; Then, the status categories can be divided according to the range size of all synchronization update rates. For example: normal, loaded, overloaded, that is: all synchronization update rates are sorted from largest to smallest to obtain a synchronization update rate sequence, and the synchronization update rate sequence is evenly divided into three sections, and then the synchronization update status of the corresponding receiving nodes in the three sections are respectively regarded as the normal state, the loaded state, and the overloaded state. It should be noted that when evenly dividing the synchronization update rate sequence, if the number of synchronization update rates is not a multiple of 3, resulting in uneven division, the excessive synchronization update rates between sections can be repeatedly divided. For example, sections: [A, B], [C, D], [E, F], [G] can be divided into sections: [A, B, C], [C, D, E], [E, F, G].

[0098] It should be noted that the synchronization update status in this application refers to the working status of each receiving node when synchronously receiving required data. The synchronization update status is divided into three categories, namely: normal state, loaded state, overloaded state. The synchronization update status can be used to monitor the working conditions of each receiving node, which is beneficial to discovering receiving nodes in the loaded or overloaded state, thereby facilitating the scheduling of receiving nodes and improving the synchronous concurrency rate of required data.

[0099] In some embodiments, referring to Figure 3 as shown, this figure is a schematic flowchart of determining the synchronization state entropy shown in some embodiments of this application. In this embodiment, the state authentication of the synchronization process of required data is performed through each synchronization update status, and the synchronization state entropy of all receiving nodes can be obtained by the following steps:

[0100] In step 1041, the state entropy of each receiving node is determined according to the synchronization update status of each receiving node;

[0101] In step 1042, the corresponding competition margin when each receiving node synchronously receives required data is obtained, and then the competition probability when the corresponding receiving node synchronously receives required data is determined through the competition margin corresponding to each receiving node;

[0102] In step 1043, the condition matching of the synchronization process of required data is performed by the state entropy and competition probability of each receiving node, and then the synchronization state entropy of all receiving nodes is obtained.

[0103] When specifically implemented, determining the state entropy of each receiving node according to the synchronous update state of each receiving node can be achieved by the following method, that is: for each receiving node, the probability that the receiving node is in a normal state, a load state, or an overload state among all receiving nodes is used as the state probability of the receiving node, and then the product of the logarithmic function value of the state probability with base 2 and the state probability is used as the state entropy of the receiving node, thereby obtaining the state entropy of all receiving nodes. Among them, the state entropy is an index for measuring the uncertainty of the receiving node being in a certain synchronous update state.

[0104] In addition, obtaining the corresponding competition margin when each receiving node synchronously receives the required data, and then determining the competition probability when the corresponding receiving node synchronously receives the required data through the competition margin of each receiving node can be achieved by the following method, that is: the competition margin is an index for measuring the ability of the receiving node to process tasks during the synchronization process of the required data. For each receiving node, the number of indexing times of the receiving node under the competition margin is obtained, and the ratio of the number of indexing times to the total number of indexing times of all receiving nodes under this competition margin is used as the competition probability of the receiving node. The competition probability is the probability that the receiving node indexes the required data under the condition of the competition margin.

[0105] When specifically implemented, conditional matching is performed on the synchronization process of the required data by the state entropy and competition probability of each receiving node, and then obtaining the synchronous state entropy of all receiving nodes can be achieved by the following method, that is: First, for each receiving node, the product of the state entropy of the receiving node and the corresponding competition probability can be used as the conditional entropy when the receiving node synchronously receives the required data, thereby obtaining the conditional entropy when all receiving nodes synchronously receive the required data. Among them, the conditional entropy represents the uncertainty of the receiving node being able to reach the condition of the normal synchronization state when the competition state is uncertain. The lower the conditional entropy, the more stable the state of the receiving node, and the higher the conditional entropy, the more chaotic the state of the receiving node; Then, the weighted sum value of all conditional entropies is used as the synchronous state entropy of all receiving nodes. Among them, the weights for weighted summing all conditional entropies can be achieved by the following method, that is: the total delay of indexing the required data for each receiving node (that is, the sum of all indexing delays) is obtained, and the ratio of the total delay of each receiving node is used as the weight of the corresponding receiving node.

[0106] It should be noted that the synchronization state entropy in this application represents a measure of the uncertainty of the conditional states of all receiving nodes when synchronously receiving required data. The larger the synchronization state entropy, the higher the condition for all receiving nodes to achieve synchronous reception of required data; the smaller the synchronization state entropy, the lower the condition for all receiving nodes to achieve synchronous reception of required data. The unstable receiving nodes can be identified through the synchronization state entropy, and the unstable receiving nodes in the system can be coordinately configured based on the synchronization state entropy, which is beneficial to optimizing the synchronization efficiency of all receiving nodes and facilitating the synchronous concurrency of receiving nodes.

[0107] In step 105, the configuration information of all receiving nodes is synchronously optimized according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, the synchronization priorities of each receiving node are adjusted according to the configuration information on the optimized configuration curve.

[0108] In some embodiments, the synchronous optimization of the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes can be implemented by the following steps:

[0109] Obtain the conditional entropy of each receiving node when synchronously receiving required data;

[0110] Determine the optimization order of each receiving node according to the conditional entropy of each receiving node and the synchronization state entropy;

[0111] Optimize the configuration information of all receiving nodes through the optimization order, the synchronization state entropy, and each synchronization scheduling value in the synchronization scheduling sequence, and monitor the configuration status of all receiving nodes to obtain the optimized configuration curve of all receiving nodes.

[0112] It should be noted that the optimization configuration of the process of receiving required data by all receiving nodes in this application means optimizing the process of receiving required data by all receiving nodes into synchronous reception. In this application, each receiving node can be coordinately configured through the synchronization scheduling sequence and the synchronization state entropy to reduce the mutual influence between each receiving node and improve the synchronization efficiency.

[0113] When specifically implemented, determining the optimization order of each receiving node according to the conditional entropy of each receiving node and the synchronization state entropy can be implemented in the following manner, that is: respectively take the ratio of the conditional entropy corresponding to each receiving node to the synchronization state entropy as the conditional proportion of the corresponding receiving node, and take the sequence obtained by arranging all the conditional proportions from large to small as the optimization order when optimizing the configuration of all receiving nodes.

[0114] In addition, the configuration information of all receiving nodes is optimized by the optimization order, the synchronization state entropy, and each synchronization scheduling value in the synchronization scheduling sequence, and the configuration status of all receiving nodes is monitored. The optimized configuration curve of all receiving nodes can be implemented in the following manner, that is: all receiving nodes can be synchronized and coordinated in the order of the optimization order, that is: according to the optimization order, the synchronization scheduling value and the synchronization state entropy of each receiving node are written into the node configuration instruction. For example: in the writing script of the configuration instruction of each node, the synchronization scheduling value is used to reduce the receiving timestamp of the receiving node, and the synchronization state entropy is used to adjust the resource occupancy rate of the receiving node. Then, each receiving node is configured through each node configuration instruction, so that all receiving nodes synchronously receive the required data. Then, the written node configuration instructions are used to configure each receiving node respectively, and the Zabbix tool in the prior art is used to monitor the configuration status (resource occupancy rate) of all receiving nodes. Then, the optimized configuration curve of all receiving nodes when receiving the required data is drawn through Zabbix. Among them, the optimized configuration curve is a dynamic curve used to monitor the receiving situation and delay status of all receiving nodes when receiving the required data. The optimized configuration curve includes the configuration information of all receiving nodes, and the configuration information includes the configuration instruction, which will not be elaborated here.

[0115] It should be noted that in this application, when the optimized configuration curve reaches the smooth standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve, that is: the optimized configuration curve can be dynamically adjusted by adjusting the proportion of the synchronization scheduling value and the synchronization state entropy in the configuration instruction. When the optimized configuration curve reaches the smooth standard, all receiving nodes are controlled by the configuration instruction in the configuration information on the optimized configuration curve, and the synchronization priority is assigned to the receiving nodes. Based on the assigned synchronization priority, the synchronization of the required data on multiple receiving nodes is realized. If the optimized configuration curve does not reach the smooth standard, the proportion of the synchronization scheduling value and the synchronization state entropy in the configuration instruction is continuously adjusted until the optimized configuration curve reaches the smooth standard. Among them, the smooth standard means that the tangent slope of the optimized configuration curve is 0; in addition, by adjusting the synchronization priority of all receiving nodes, the synchronization delay of each receiving node can be reduced, which is beneficial to improving the efficiency of the synchronization concurrency of the required data. The synchronization priority refers to the different processing orders and resource occupancy rates assigned by the system to the receiving nodes.

[0116] On the other hand, in some embodiments, this application provides a customer demand management system, which includes a data synchronization unit. Refer to Figure 4 , this figure is a schematic structural diagram of the data synchronization unit in some embodiments of this application. The data synchronization unit 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described as follows:

[0117] An obtaining module 401, in the present application, the obtaining module 401 is mainly used to obtain the synchronization record of demand data to obtain demand synchronization data;

[0118] A processing module 402, in the present application, the processing module 402 is used to determine the competition margin of each receiving node when synchronously receiving demand data according to the configuration information of each receiving node during the synchronization process of the demand data and the allocation log in the demand synchronization data;

[0119] In the present application, the processing module 402 is further used to audit the demand data synchronously received by all receiving nodes to obtain the audit volume during the demand data synchronization, and then determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving the demand data from the audit volume and each competition margin;

[0120] In the present application, the processing module 402 is further used to determine the synchronization update status of each receiving node based on the index record of the demand data of each receiving node, and then perform status authentication on the synchronization process of the demand data through each synchronization update status to obtain the synchronization state entropy of all receiving nodes;

[0121] An execution module 403, in the present application, the execution module 403 is mainly used to synchronously optimize the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain an optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

[0122] In addition, the present application further provides a computer device, the computer device includes a memory and a processor, the memory stores code, and the processor is configured to obtain the code and execute the above data synchronization method for a customer demand management system.

[0123] In some embodiments, refer to Figure 5 , this figure is a schematic structural diagram of a computer device for implementing the data synchronization method for a customer demand management system according to some embodiments of the present application. The data synchronization method for a customer demand management system in the above embodiments can be implemented by Figure 5 The computer device shown, the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0124] The processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0125] The communication bus 502 can be used to transfer information among the above components.

[0126] The memory 503 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but not limited thereto. The memory 503 can exist independently and be connected to the processor 501 through the communication bus 502. The memory 503 can also be integrated with the processor 501.

[0127] Among them, the memory 503 is used to store the program code for executing the solution of this application, and is controlled by the processor 501 for execution. The processor 501 is used to execute the program code stored in the memory 503. The program code can include one or more software modules. The data synchronization method for the customer demand management system in the above embodiments can be implemented by one or more software modules in the program code of the processor 501 and the memory 503.

[0128] The communication interface 504, using any device such as a transceiver, is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0129] In a specific implementation, as an embodiment, the computer device can include multiple processors, and each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0130] The computer device described above may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0131] In addition, the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the data synchronization method for the customer demand management system described above is implemented.

[0132] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0133] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A data synchronization method for a customer demand management system, characterized in that, including the following steps: Obtain the synchronization record of the requirement data to get the requirement synchronization data; Determine the competition margin of each receiving node when synchronously receiving the requirement data according to the configuration information of each receiving node during the synchronization process of the requirement data and the allocation log in the requirement synchronization data; Audit the requirement data synchronously received by all receiving nodes to obtain the audit volume during the requirement data synchronization, and then determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving the requirement data based on the audit volume and each competition margin; Determine the synchronization update status of each receiving node based on the index record of the requirement data of each receiving node, and then perform status authentication on the synchronization process of the requirement data through each synchronization update status to obtain the synchronization state entropy of all receiving nodes; Synchronously optimize the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve meets the smoothing standard, adjust the synchronization priority of each receiving node according to the configuration information on the optimized configuration curve.

2. The method according to claim 1, characterized in that, Determining the competition margin of each receiving node when synchronously receiving the requirement data according to the configuration information of each receiving node during the synchronization process of the requirement data and the allocation log in the requirement synchronization data specifically includes: Obtain the configuration information of each receiving node during the synchronization process of the requirement data; Determine the delay characteristics of each receiving node during the requirement data synchronization process according to the configuration information of each receiving node; Extract the allocation logs of all receiving nodes during the requirement data synchronization from the requirement synchronization data; Determine the task allocation list of all receiving nodes during the requirement data synchronization through the allocation log; Determine the competition margin of each receiving node when synchronously receiving the requirement data based on the task allocation list and the delay characteristics of each receiving node during the requirement data synchronization process.

3. The method according to claim 1, characterized in that Auditing the requirement data synchronously received by all receiving nodes to obtain the audit volume during the requirement data synchronization specifically includes: Determine the attribute entropy of each receiving node; Obtain the configuration information of each receiving node, and then determine the configuration score of each receiving node according to the configuration information of each receiving node; Determine the audit coefficient of each receiving node through the configuration score and the attribute entropy of each receiving node; Obtain the data information flow of each receiving node synchronously receiving the requirement data; Perform information audit on the data information flow of each receiving node based on the audit coefficient of each receiving node, and then obtain the audit volume during the requirement data synchronization.

4. The method according to claim 1, characterized in that, Determining the synchronization scheduling sequence of all receiving nodes when synchronously receiving the requirement data based on the audit volume and each competition margin specifically includes: Determine the coordination degree of each receiving node when synchronously receiving the requirement data according to the audit volume; Perform collaborative scheduling on the process of synchronously receiving the requirement data through the coordination degree and the competition margin of each receiving node to obtain the synchronization scheduling value of each receiving node; Determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving the requirement data based on the synchronization scheduling value of each receiving node.

5. The method according to claim 1, wherein Determining the synchronous update status of each receiving node based on the index records of the demand data by each receiving node specifically includes: Obtaining the index records of the demand data by each receiving node; Extracting the synchronous delay of each receiving node from each index record; Obtaining the data information flow of each receiving node for synchronously receiving the demand data; Determining the synchronous update rate of each receiving node through the synchronous delay and the data information flow of each receiving node, and further determining the synchronous update status of each receiving node.

6. The method according to claim 1, characterized in that, Synchronously optimizing the configuration information of all receiving nodes according to the synchronous scheduling sequence and the synchronous state entropy to obtain the optimized configuration curve of all receiving nodes specifically includes: Obtaining the conditional entropy when each receiving node synchronously receives the demand data; Determining the optimization order of each receiving node according to the conditional entropy of each receiving node and the synchronous state entropy; Optimizing the configuration information of all receiving nodes through the optimization order, the synchronous state entropy, and each synchronous scheduling value in the synchronous scheduling sequence, and monitoring the configuration status of all receiving nodes to obtain the optimized configuration curve of all receiving nodes.

7. The method according to claim 1, characterized in that, The allocation log is a database log.

8. A customer demand management system, the customer demand management system includes a data synchronization unit, characterized in that, The data synchronization unit includes: An obtaining module, configured to obtain the synchronization record of the demand data to obtain the demand synchronization data; A processing module, configured to determine the competition margin when each receiving node synchronously receives the demand data according to the configuration information of each receiving node during the synchronization process of the demand data and the allocation log in the demand synchronization data; The processing module is further configured to audit the demand data synchronously received by all receiving nodes to obtain the audit amount during the demand data synchronization, and further determine the synchronous scheduling sequence of all receiving nodes when synchronously receiving the demand data from the audit amount and each competition margin; The processing module is further configured to determine the synchronous update status of each receiving node based on the index records of the demand data by each receiving node, and further perform status authentication on the synchronization process of the demand data through each synchronous update status to obtain the synchronous state entropy of all receiving nodes; An execution module, configured to synchronously optimize the configuration information of all receiving nodes according to the synchronous scheduling sequence and the synchronous state entropy to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, adjust the synchronous priority of each receiving node according to the configuration information on the optimized configuration curve.

9. A computer device, the computer device comprising a memory and a processor, the memory storing code, characterized in that, The processor is configured to obtain the code and execute the data synchronization method for the customer demand management system according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the data synchronization method for the customer demand management system according to any one of claims 1 to 7.

Images