A data synchronization method and related equipment for a customer demand management system

Abstract

This application provides a data synchronization method and related equipment for a customer demand management system. The method determines the contention margin of receiving nodes through configuration information and allocation logs, then audits the demand data synchronously received by the receiving nodes to obtain the audit quantity during demand data synchronization. The audit quantity and contention margin determine the synchronization scheduling sequence of the receiving nodes during synchronous demand data reception. Furthermore, the synchronization update status of the receiving nodes is determined through the receiving node index records, and the synchronization update status is used to authenticate the demand data synchronization process, obtaining a synchronization state entropy. Finally, the configuration information of the receiving nodes is optimized based on the synchronization scheduling sequence and the synchronization state entropy to obtain an optimized configuration curve. The synchronization priority of the receiving nodes is then adjusted based on the configuration information on the optimized configuration curve. Using the scheme of this application, the synchronization priority of receiving nodes can be optimized, thereby improving the audit efficiency of the customer demand management system during data synchronization.

Description

Technical Field

[0001] This application relates to the field of data synchronization technology, and more specifically, to a data synchronization method and related equipment for a customer demand management system. Background Technology

[0002] Data synchronization refers to the process of ensuring that multiple nodes in a distributed system receive the same data information at the same time. Data synchronization has a wide range of applications, including database synchronization, file synchronization, application data synchronization, and other data synchronization. Data synchronization usually involves data replication, transmission, and consistency verification, and is an important technology to ensure system stability and data accuracy.

[0003] Data synchronization in a customer demand management system refers to the process of ensuring consistency and real-time updates of customer demand information across different users. Existing customer demand management systems typically rely on a multi-node distributed architecture. However, under concurrent resource conditions across multiple nodes, the synchronization competition among receiving nodes is intense when requesting data updates. The allocation and scheduling of resources among nodes are difficult to balance, leading to data discrepancies and reception delays when different receiving nodes synchronously receive demand data. Therefore, how to optimize the scheduling of receiving nodes to improve the audit efficiency of customer demand management systems during data synchronization has become a challenge for the industry. Summary of the Invention

[0004] This application provides a data synchronization method and related equipment for a customer demand management system, which can optimize the scheduling of synchronization priority of receiving nodes, thereby improving the audit efficiency of the customer demand management system during data synchronization.

[0005] In a first aspect, this application provides a data synchronization method for a customer demand management system, comprising the following steps:

[0006] Obtain the synchronized records of the demand data to get the demand synchronization data;

[0007] The contention margin of each receiving node when synchronously receiving the demand data is determined based on the configuration information of each receiving node during the synchronization process and the allocation log in the demand synchronization data.

[0008] Audit the required data received synchronously by all receiving nodes to obtain the audit amount when the required data is synchronized. Then, the synchronization scheduling sequence of all receiving nodes when synchronously receiving the required data is determined by the audit amount and each contention margin.

[0009] Based on the index record of the demand data of each receiving node, the synchronization update status of each receiving node is determined, and then the synchronization process of the demand data is authenticated through each synchronization update status to obtain the synchronization state entropy of all receiving nodes.

[0010] Based on the synchronization scheduling sequence and the synchronization state entropy, the configuration information of all receiving nodes is synchronized and optimized to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothness standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

[0011] In some embodiments, determining the contention margin of each receiving node when synchronously receiving demand data, based on the configuration information of each receiving node during the synchronization process of demand data and the allocation log in the demand synchronization data, specifically includes:

[0012] Obtain the configuration information of each receiving node during the synchronization process to obtain the required data;

[0013] The latency characteristics of each receiving node during the required data synchronization process are determined based on the configuration information of each receiving node.

[0014] Extract the allocation logs of all receiving nodes during the demand data synchronization from the demand data synchronization data;

[0015] The task allocation list of all receiving nodes during the required data synchronization is determined through the allocation log.

[0016] The contention margin of each receiving node when synchronously receiving the required data is determined by the task allocation list and the latency characteristics of each receiving node during the required data synchronization process.

[0017] In some embodiments, auditing the required data received synchronously by all receiving nodes to obtain the audit quantity when the required data is synchronized specifically includes:

[0018] Determine the attribute entropy of each receiving node;

[0019] The configuration information of each receiving node is obtained, and then the configuration score of each receiving node is determined based on the configuration information of each receiving node.

[0020] The audit coefficient of each receiving node is determined by the configuration score and attribute entropy of each receiving node;

[0021] Obtain the data information stream of each receiving node synchronously receiving the required data;

[0022] Based on the audit coefficient of each receiving node, the data information flow of each receiving node is audited to obtain the audit quantity when data synchronization is required.

[0023] In some embodiments, the synchronization scheduling sequence of all receiving nodes when determining the synchronous reception requirement data based on the audit quantity and various contention margins specifically includes:

[0024] The coordination degree of each receiving node when synchronously receiving required data is determined based on the audit quantity.

[0025] The process of synchronously receiving data is coordinated and scheduled by the coordination degree and contention margin of each receiving node, so as to obtain the synchronization scheduling value of each receiving node.

[0026] The synchronization scheduling sequence of all receiving nodes is determined based on the synchronization scheduling value of each receiving node when synchronously receiving the required data.

[0027] In some embodiments, determining the synchronization update status of each receiving node based on its index record of the required data specifically includes:

[0028] Obtain the index record of the required data for each receiving node;

[0029] Extract the synchronization latency of each receiving node from each index record;

[0030] Obtain the data information stream of each receiving node synchronously receiving the required data;

[0031] The synchronization update rate of each receiving node is determined by the synchronization delay and data information flow of each receiving node, thereby determining the synchronization update status of each receiving node.

[0032] In some embodiments, the configuration information of all receiving nodes is synchronized and optimized based on the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curves of all receiving nodes, specifically including:

[0033] Obtain the conditional entropy of each receiving node when it synchronously receives the required data;

[0034] The optimization order of each receiving node is determined based on the conditional entropy of each receiving node and the synchronization state entropy.

[0035] The configuration information of all receiving nodes is optimized by using the optimization order, the synchronization state entropy, and the synchronization scheduling values ​​in the synchronization scheduling sequence, and the configuration status of all receiving nodes is monitored to obtain the optimized configuration curves of all receiving nodes.

[0036] In some embodiments, the allocation log is a database log.

[0037] Secondly, this application provides a customer demand management system, which includes a data synchronization unit, the data synchronization unit comprising:

[0038] The acquisition module is used to acquire the synchronization records of the demand data and obtain the demand synchronization data.

[0039] The processing module is used to determine the contention margin of each receiving node when synchronously receiving the demand data based on the configuration information of each receiving node during the synchronization process and the allocation log in the demand synchronization data.

[0040] The processing module is also used to audit the demand data received synchronously by all receiving nodes, obtain the audit amount when the demand data is synchronized, and then determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data based on the audit amount and each contention margin.

[0041] The processing module is also used to determine the synchronization update status of each receiving node based on the index record of the demand data of each receiving node, and then perform state authentication on the synchronization process of the demand data through each synchronization update status to obtain the synchronization state entropy of all receiving nodes.

[0042] The execution module is used to perform synchronization optimization on the configuration information of all receiving nodes based on the synchronization scheduling sequence and the synchronization state entropy, to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

[0043] Thirdly, this application provides a computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the data synchronization method for a customer demand management system described above.

[0044] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned data synchronization method for a customer demand management system.

[0045] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0046] The data synchronization method and related equipment for a customer demand management system provided in this application firstly acquire the synchronization records of demand data to obtain demand synchronization data; secondly, based on the configuration information of each receiving node during the synchronization process of demand data and the allocation log in the demand synchronization data, the contention margin of each receiving node when synchronously receiving demand data is determined; next, the demand data synchronously received by all receiving nodes is audited to obtain the audit quantity during demand data synchronization, and then the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data is determined by the audit quantity and each contention margin; then, the synchronization update status of each receiving node is determined based on the index record of demand data for each receiving node, and then the synchronization process of demand data is authenticated through each synchronization update status to obtain the synchronization state entropy of all receiving nodes; finally, the configuration information of all receiving nodes is synchronously optimized according to the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

[0047] Therefore, this application optimizes the synchronization of demand data by adjusting the synchronization priority of all receiving nodes based on the configuration information on the configuration curve. First, a synchronization scheduling sequence is formed using the synchronization scheduling values ​​of each receiving node. These values ​​measure the scheduling priority of each receiving node when synchronously receiving demand data. Auditing the demand data received by different receiving nodes based on these synchronization scheduling values ​​helps determine the scheduling values ​​and optimization order of all receiving nodes during demand data synchronization. This, in turn, optimizes the receiving nodes using the determined scheduling values ​​and optimization order, thereby improving the accuracy of demand data synchronization. Second, by determining the synchronization state entropy, the synchronization priority of all receiving nodes when synchronously receiving demand data is clarified. The synchronization conditions facilitate the identification of unstable receiving nodes during the data synchronization process. Furthermore, the system allows for conditional configuration of unstable receiving nodes based on synchronization state entropy, thereby optimizing the concurrency efficiency of all receiving nodes. Finally, the configuration information of all receiving nodes is optimized based on the synchronization scheduling sequence and synchronization state entropy to obtain an optimized configuration curve. The synchronization priority of receiving nodes is adjusted using the configuration information on the optimized configuration curve, which helps reduce resource occupancy during data synchronization and promotes high-concurrency synchronous reception of data at receiving nodes. In summary, this application's solution can optimize the scheduling of receiving node synchronization priorities, thereby improving the audit efficiency of the customer demand management system during data synchronization. Attached Figure Description

[0048] Figure 1 This is an exemplary flowchart of a data synchronization method for a customer demand management system according to some embodiments of this application;

[0049] Figure 2 This is a flowchart illustrating the process of determining the contention margin according to some embodiments of this application;

[0050] Figure 3 This is a flowchart illustrating the process of determining the synchronization state entropy according to some embodiments of this application;

[0051] Figure 4 This is a schematic diagram of the structure of the data synchronization unit in some embodiments of this application;

[0052] Figure 5 This is a schematic diagram of the structure of a computer device implementing a data synchronization method for a customer demand management system, according to some embodiments of this application. Detailed Implementation

[0053] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] refer to Figure 1 The figure is an exemplary flowchart of a data synchronization method for a customer demand management system according to some embodiments of this application. The data synchronization method 100 for a customer demand management system mainly includes the following steps:

[0055] In step 101, the synchronization record of the demand data is obtained to obtain the demand synchronization data.

[0056] In specific implementation, the synchronization records of demand data are obtained. The demand synchronization data can be obtained in the following way: the synchronization records of target customer demand data within a specified time period are obtained from the database of the customer demand management system, and the set of all synchronization records is taken as the target customer demand synchronization data. The specified time period is from the current time to the past week. The synchronization records include: creation time, creator, demand title, demand description, demand priority, demand processing status, and allocation log. In addition, the allocation log is a database log, which represents the set of information at each receiving node when the demand data in the customer demand management system is synchronized.

[0057] It should be noted that the database of the customer demand management system in this application is a distributed database composed of two or more data nodes. The data nodes include master nodes and receiving nodes. The master node is the creation node for synchronizing records. When the demand data is synchronized, each receiving node corresponds to a different user of the customer demand management system. The different users include: customers, customer service, R&D team, data analysts and other system users.

[0058] In step 102, the contention margin of each receiving node when synchronously receiving the demand data is determined based on the configuration information of each receiving node during the synchronization process of the demand data and the allocation log in the demand synchronization data.

[0059] In some embodiments, reference Figure 2 As shown in the figure, this is a flowchart illustrating the process of determining contention margin in some embodiments of this application. In this embodiment, determining the contention margin of each receiving node when synchronously receiving the demand data based on the configuration information of each receiving node during the synchronization process and the allocation log in the demand synchronization data can be achieved through the following steps:

[0060] In step 1021, the configuration information of each receiving node during the synchronization process of the required data is obtained;

[0061] In step 1022, the latency characteristics of each receiving node during the required data synchronization process are determined based on the configuration information of each receiving node;

[0062] In step 1023, the allocation logs of all receiving nodes during the demand data synchronization are extracted from the demand synchronization data;

[0063] In step 1024, the task allocation list of all receiving nodes during the required data synchronization is determined through the allocation log;

[0064] In step 1025, the contention margin of each receiving node when synchronously receiving the demand data is determined by the task allocation list and the latency characteristics of each receiving node during the demand data synchronization process.

[0065] In specific implementation, obtaining the configuration information of each receiving node during the demand data synchronization process can be achieved in the following way: During demand data synchronization, an interface configuration file is loaded from the customer demand management system, and the configuration information of each receiving node is obtained from the interface configuration file. The suffix of the interface configuration file can be ".yaml". The configuration information includes the configuration instructions, hardware configuration, network configuration, and processing capabilities of the corresponding node. Determining the latency characteristics of each receiving node during the demand data synchronization process based on the configuration information of each receiving node can be achieved in the following way: The processing capability of the receiving node includes processing time, data size, and disk load. For each receiving node, the latency in the disk load of the receiving node can be used as the latency characteristic of the receiving node during the demand data synchronization process, thereby obtaining the latency characteristics of all receiving nodes during the demand data synchronization process. The disk load of the receiving node includes IOPS (input / output operations per second), throughput, and latency.

[0066] In addition, in specific implementation, the allocation logs of all receiving nodes during the synchronization of the demand data can be extracted from the demand synchronization data in the following way: the allocation logs refer to the collection of information at each receiving node during the synchronization of the demand data. The information at each receiving node includes: node identifier, task identifier, task allocation time, task status, synchronization time, and node configuration parameters. A database query statement can be written to extract the information at each receiving node, and the collection of all information is used as the allocation logs of all receiving nodes during the synchronization of the demand data.

[0067] In specific implementation, determining the task allocation list of all receiving nodes during demand data synchronization through the allocation log can be achieved in the following way: The task allocation time ratio of each receiving node (i.e., the ratio of the task allocation time of each receiving node to the sum of all task allocation times) is arranged in ascending order to form the task allocation list of all receiving nodes during demand data synchronization. Determining the contention margin of each receiving node when synchronously receiving demand data using the task allocation list and the latency characteristics of each receiving node during the demand data synchronization process can be achieved in the following way: The task allocation list includes the task allocation time ratio of each receiving node, and the product of the task allocation time ratio and the corresponding latency characteristic can be used as the contention margin of the corresponding receiving node when synchronously receiving demand data.

[0068] It should be noted that the contention margin in this application is an indicator that measures the ability of receiving nodes to process tasks during the data synchronization process. The contention margin reflects the elasticity of each receiving node when processing multiple tasks simultaneously during the data synchronization process. That is, receiving nodes with higher contention margins have stronger ability to handle multiple tasks concurrently, while receiving nodes with lower contention margins have limited ability to process multiple tasks simultaneously, which can easily lead to longer latency.

[0069] In step 103, the required data received synchronously by all receiving nodes is audited to obtain the audit quantity when the required data is synchronized. Then, the synchronization scheduling sequence of all receiving nodes when synchronously receiving the required data is determined by the audit quantity and each contention margin.

[0070] In some embodiments, auditing the required data synchronously received by all receiving nodes to obtain the audit quantity when the required data is synchronized can be achieved by the following steps:

[0071] Determine the attribute entropy of each receiving node;

[0072] The configuration information of each receiving node is obtained, and then the configuration score of each receiving node is determined based on the configuration information of each receiving node.

[0073] The audit coefficient of each receiving node is determined by the configuration score and attribute entropy of each receiving node;

[0074] Obtain the data information stream of each receiving node synchronously receiving the required data;

[0075] Based on the audit coefficient of each receiving node, the data information flow of each receiving node is audited to obtain the audit quantity when data synchronization is required.

[0076] In practical implementation, the attribute entropy of each receiving node can be determined in the following way: Attribute entropy is an indicator that measures the stability of the state attributes of a receiving node. The higher the attribute entropy, the more complex the state attributes of the receiving node and the greater the uncertainty of the receiving node's state. The lower the attribute entropy, the more stable the state attributes of the receiving node and the greater the predictability of the receiving node's state. State attributes include: processing capacity, network latency, disk I / O performance, data processing time, and load balancing. For each receiving node, the information content of each attribute of the receiving node can be described by information entropy, and then the sum of the information entropy of each attribute can be used as the attribute entropy of the receiving node, thereby obtaining the attribute entropy of all receiving nodes.

[0077] In addition, in specific implementation, obtaining the configuration information of each receiving node and then determining the configuration score of each receiving node based on the configuration information of each receiving node can be achieved in the following way: the configuration information of each receiving node includes the processing time, data size, and disk load of the corresponding receiving node. For each receiving node, the processing time, data size, and disk load of the receiving node are standardized using the standard scores in the prior art, and the weighted sum of the standardized processing time, data size, and disk load is used as the configuration score of the receiving node, thereby obtaining the configuration scores of all receiving nodes. Among them, the weighted sum of the standardized processing time, data size, and disk load can be adaptively set according to the actual proportion of the standardized processing time, data size, and disk load of each interface.

[0078] In specific implementation, the audit coefficient of each receiving node can be determined by the configuration score and attribute entropy of each receiving node in the following way: The audit coefficient is a parameter that measures the audit capability of the receiving node. The configuration score and attribute entropy of all receiving nodes can be input into the initialized regression analysis model for model training. For each receiving node, the configuration score and attribute entropy of the receiving node are respectively input into the regression analysis model for prediction. The value of the linear combination output by the regression analysis model is then used as the audit coefficient of the receiving node, thereby obtaining the audit coefficient of each receiving node.

[0079] In addition, the data information stream of each receiving node synchronously receiving the required data can be obtained in the following way: the Wireshark network analysis tool can be used to capture the data stream at each receiving node when the required data is synchronized, and the set of all data streams captured at each receiving node is taken as the data information stream of the corresponding receiving node synchronously receiving the required data. Other network analysis tools can also be used for capture in other embodiments, which is not limited here.

[0080] In specific implementation, the audit amount for data information flow of each receiving node is obtained by auditing the audit coefficient of each receiving node, which can be achieved in the following way: First, the hash value of the data information flow of each node is extracted using the existing hash function, and then all receiving nodes are hash-sorted using all hash values; then, for each receiving node, the proportion of the receiving node priority obtained after hash sorting is used as the weight value of the receiving node, the product of the weight value and the hash value is used as the weighted hash value of the receiving node, and the product of the weighted hash value and the information content of the data information flow is used as the audit value of the receiving node, thus obtaining the audit value of each receiving node; finally, the standard deviation of all audit values ​​is used as the audit amount for data synchronization.

[0081] It should be noted that the audit quantity in this application is an indicator for measuring the degree of uniformity of the demand information received synchronously by different receiving nodes. That is, the larger the audit quantity, the more uniform the demand information received synchronously by different receiving nodes; the smaller the audit quantity, the greater the deviation of the demand information received synchronously by different receiving nodes. The audit quantity can also represent the data integrity of the demand data received by all receiving interfaces. A higher audit quantity indicates that the data receiving status of each receiving node is relatively stable and the data synchronization quality is high. A lower audit quantity indicates that there are abnormalities in the data receiving of the receiving nodes. Anomalies can be filtered based on the audit value of each interface, and the abnormal receiving nodes can be optimized and scheduled, thereby improving the consistency of the demand data received by the receiving nodes.

[0082] In some embodiments, the synchronization scheduling sequence of all receiving nodes when determining the synchronous reception requirement data based on the audit quantity and various contention margins can be achieved by the following steps:

[0083] The coordination degree of each receiving node when synchronously receiving required data is determined based on the audit quantity.

[0084] The process of synchronously receiving data is coordinated and scheduled by the coordination degree and contention margin of each receiving node, so as to obtain the synchronization scheduling value of each receiving node.

[0085] The synchronization scheduling sequence of all receiving nodes is determined based on the synchronization scheduling value of each receiving node when synchronously receiving the required data.

[0086] In specific implementation, the coordination degree of each receiving node when synchronously receiving required data can be determined based on the audit quantity in the following way: For each receiving node, the absolute difference between the audit value of the receiving node and the audit quantity is taken as the audit difference of the receiving node. Then, the audit quantity is used as the normalization standard to normalize the audit difference of the receiving node, so as to obtain the normalized audit difference (i.e., audit difference / audit quantity). As a preferred embodiment, the coordination degree of the receiving node when synchronously receiving required data can be calculated using the following formula: Coordination degree = 1 / (1 + normalized audit difference), thereby obtaining the coordination degree of each receiving node when synchronously receiving required data.

[0087] Furthermore, in specific implementation, the process of synchronously receiving the required data is coordinated and scheduled based on the coordination degree and contention margin of each receiving node. The synchronization scheduling value for each receiving node can be obtained in the following way: The coordination degree and contention margin of all receiving nodes are normalized using the max-min normalization method in existing technology, and the latency of all receiving nodes during synchronously receiving the required data is obtained. The latency is then plotted in a statistical graph. For each receiving node, its coordination degree and contention margin are weighted and summed, with each weight set to 0.5. The synchronization scheduling value for each node is then adjusted accordingly. The weights of coordination degree and contention margin are adjusted (increased and decreased), and the sum of the weights of coordination degree and contention margin is guaranteed to be 1. When the delay in the statistical graph tends to be minimum and stable, the weighted sum of the coordination degree and contention margin of the receiving node is used as the synchronization scheduling value of the receiving node, and thus the synchronization scheduling value of each receiving node is obtained. The synchronization scheduling sequence of all receiving nodes when determining the synchronous receiving demand data based on the synchronization scheduling value of each receiving node can be implemented in the following way: the sequence of all synchronization scheduling values ​​is randomly composed as the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data.

[0088] It should be noted that the synchronization scheduling sequence in this application refers to a sequence composed of the synchronization scheduling values ​​of each receiving node. The synchronization scheduling value is used to measure the scheduling priority of the receiving node when synchronously receiving the required data. The higher the synchronization scheduling value, the higher the priority of the receiving node in the synchronization process, and the greater the competition among the receiving nodes. Conversely, the higher the synchronization scheduling value, the more delayed the receiving node is in the synchronization process, and the less competitive it is. The synchronization scheduling sequence can ensure that each receiving node can process the required data in the optimal order during the synchronization process, thereby improving the consistency of the required data during synchronization.

[0089] In step 104, the synchronization update status of each receiving node is determined based on the index record of the demand data of each receiving node, and then the synchronization process of the demand data is authenticated through each synchronization update status to obtain the synchronization state entropy of all receiving nodes.

[0090] In some embodiments, determining the synchronization update status of each receiving node based on its index record of the required data can be achieved using the following steps:

[0091] Obtain the index record of the required data for each receiving node;

[0092] Extract the synchronization latency of each receiving node from each index record;

[0093] Obtain the data information stream of each receiving node synchronously receiving the required data;

[0094] The synchronization update rate of each receiving node is determined by the synchronization delay and data information flow of each receiving node, thereby determining the synchronization update status of each receiving node.

[0095] In specific implementation, obtaining the index record of each receiving node for the required data can be achieved in the following way: obtain the operation log of each receiving node, and extract the corresponding index record of the receiving node for the required data from each operation log. The operation log refers to the log file generated by each receiving node in the process of processing the required data. The operation log records the receiving node's receipt, processing, and storage of the required data. The operation log includes: timestamp, node identifier, operation type, processing time, data size, and other log information. The index record is specific information extracted from the operation log and is used to track how each receiving node processes the required data. The index record includes node identifier, operation time, and data size. The index record is helpful for analyzing the efficiency of required data synchronization.

[0096] In addition, the synchronization latency of each receiving node can be extracted from each index record in the following way: First, extract the timestamp information from the index record of each receiving node. For each receiving node, obtain the start time and end time of the required data synchronization from the timestamp information of the receiving node, and use the end time and start time as the synchronization latency of the receiving node, thereby obtaining the synchronization latency of all receiving nodes. The timestamp information refers to the specific time marked in the operation log for each operation. The timestamp information can record the time sequence of events. The synchronization latency refers to the time spent from the start of processing the required data at the receiving node to the end of processing, which is used to measure the efficiency of the receiving node in processing the required data.

[0097] In specific implementation, the synchronization update rate of each receiving node is determined by the synchronization delay and data information flow of each receiving node, and the synchronization update status of each receiving node is determined in the following way: First, the size of the data packet in the data information flow of each receiving node is taken as the data packet size of each receiving node. For each receiving node, the synchronization update rate can be calculated using the following formula: Synchronization update rate = Data packet size * Synchronization delay, thus obtaining the synchronization update rate of each receiving node; Then, the status categories can be divided according to the range of all synchronization update rates, for example: normal, load, overload. That is, sort all synchronization update rates from largest to smallest to obtain a synchronization update rate sequence, and divide the synchronization update rate sequence equally into three segments. Then, the synchronization update status of the corresponding receiving nodes in the three segments is regarded as normal state, load state, and overload state, respectively. It should be noted that when dividing the synchronization update rate sequence equally, if the number of synchronization update rates is not a multiple of 3, resulting in uneven division, the excessive synchronization update rates between segments can be repeatedly divided. For example, the segments [A,B], [C,D], [E,F], [G] can be divided into the segments [A,B,C], [C,D,E], [E,F,G].

[0098] It should be noted that the synchronous update status in this application refers to the working status of each receiving node when synchronously receiving the required data. The synchronous update status is divided into three categories: normal status, load status, and overload status. The synchronous update status can be used to monitor the working status of each receiving node, which is helpful to discover receiving nodes that are under load or overload, thereby facilitating the scheduling of receiving nodes and improving the synchronous concurrency rate of required data.

[0099] In some embodiments, reference Figure 3 As shown in the figure, this is a flowchart illustrating the process of determining the synchronization state entropy in some embodiments of this application. In this embodiment, the synchronization process of the required data is authenticated through each synchronization update state, and the synchronization state entropy of all receiving nodes can be obtained by the following steps:

[0100] In step 1041, the state entropy of each receiving node is determined based on the synchronization update state of each receiving node;

[0101] In step 1042, the contention margin of each receiving node when synchronously receiving the required data is obtained, and then the contention probability of the corresponding receiving node when synchronously receiving the required data is determined by the contention margin of each receiving node.

[0102] In step 1043, the synchronization process of the required data is conditionally matched by the state entropy and contention probability of each receiving node, thereby obtaining the synchronization state entropy of all receiving nodes.

[0103] In specific implementation, the state entropy of each receiving node can be determined based on the synchronization update state of each receiving node in the following way: for each receiving node, the probability that the receiving node is in a normal state, a load state, or an overload state among all receiving nodes is taken as the state probability of the receiving node. Then, the product of the logarithm of the state probability to the base 2 and the state probability is taken as the state entropy of the receiving node, thereby obtaining the state entropy of all receiving nodes. Here, the state entropy is an indicator that measures the uncertainty of the receiving node being in a certain synchronization update state.

[0104] In addition, the contention margin corresponding to each receiving node when synchronously receiving the required data can be obtained, and the contention probability of the corresponding receiving node when synchronously receiving the required data can be determined by the contention margin of each receiving node. The contention margin is a measure of the receiving node's ability to process tasks during the synchronization of required data. For each receiving node, the number of indexes of the receiving node under the contention margin is obtained, and the ratio of the number of indexes to the total number of indexes of all receiving nodes under the contention margin is taken as the contention probability of the receiving node. The contention probability is the probability of the receiving node indexing the required data under the contention margin condition.

[0105] In specific implementation, the synchronization state entropy of all receiving nodes is obtained by conditionally matching the state entropy and contention probability of each receiving node to the synchronization process of the demand data. This can be achieved in the following way: First, for each receiving node, the product of the state entropy of the receiving node and the corresponding contention probability can be used as the conditional entropy of the receiving node when synchronously receiving the demand data. This yields the conditional entropy of all receiving nodes when synchronously receiving the demand data. Here, the conditional entropy represents the uncertainty of whether the receiving node can achieve a normal synchronization state under uncertain contention conditions. The lower the conditional entropy, the more stable the state of the receiving node; the higher the conditional entropy, the more chaotic the state of the receiving node. Next, the weighted sum of all conditional entropies is used as the synchronization state entropy of all receiving nodes. The weights for the weighted sum of all conditional entropies can be obtained by: obtaining the total latency of each receiving node indexing the demand data (i.e., the sum of all index latencies) and using the proportion of the total latency of each receiving node as the weight of the corresponding receiving node.

[0106] It should be noted that the synchronization state entropy in this application represents a measure of the uncertainty of the conditional state of all receiving nodes when synchronously receiving the required data. The larger the synchronization state entropy, the higher the condition for all receiving nodes to synchronously receive the required data; the smaller the synchronization state entropy, the lower the condition for all receiving nodes to synchronously receive the required data. The synchronization state entropy can identify unstable receiving nodes, and the unstable receiving nodes in the system can be coordinated and configured according to the synchronization state entropy, which is beneficial to optimizing the synchronization efficiency of all receiving nodes and facilitating the synchronous concurrency of receiving nodes.

[0107] In step 105, the configuration information of all receiving nodes is optimized based on the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

[0108] In some embodiments, optimizing the configuration information of all receiving nodes based on the synchronization scheduling sequence and the synchronization state entropy to obtain the optimized configuration curves of all receiving nodes can be achieved through the following steps:

[0109] Obtain the conditional entropy of each receiving node when it synchronously receives the required data;

[0110] The optimization order of each receiving node is determined based on the conditional entropy of each receiving node and the synchronization state entropy.

[0111] The configuration information of all receiving nodes is optimized by using the optimization order, the synchronization state entropy, and the synchronization scheduling values ​​in the synchronization scheduling sequence, and the configuration status of all receiving nodes is monitored to obtain the optimized configuration curves of all receiving nodes.

[0112] It should be noted that the optimized configuration of the process of receiving the required data by all receiving nodes in this application refers to optimizing the process of receiving the required data by all receiving nodes into synchronous reception. In this application, the synchronous scheduling sequence and synchronous state entropy can be used to coordinate the configuration of each receiving node to reduce the mutual influence between the receiving nodes and improve the synchronization efficiency.

[0113] In specific implementation, the optimization order of each receiving node can be determined based on the conditional entropy of each receiving node and the synchronization state entropy in the following way: the ratio of the conditional entropy of each receiving node to the synchronization state entropy is taken as the conditional proportion of the corresponding receiving node, and the sequence obtained by arranging all the conditional proportions in descending order is taken as the optimization order when optimizing the configuration of all receiving nodes.

[0114] Furthermore, optimizing the configuration information of all receiving nodes using the optimization order, the synchronization state entropy, and the synchronization scheduling values ​​in the synchronization scheduling sequence, and monitoring the configuration status of all receiving nodes, to obtain the optimized configuration curves for all receiving nodes can be achieved in the following way: all receiving nodes can be synchronized and coordinated according to the optimization order, that is: according to the optimization order, the synchronization scheduling values ​​and synchronization state entropies of each receiving node are written into the node configuration instructions. For example, in the script for writing the configuration instructions for each node, the synchronization scheduling values ​​are used to reduce the receiving timestamp of the receiving node, and the synchronization state entropy is used to adjust the resource occupancy rate of the receiving node. By configuring each receiving node using node configuration instructions, all receiving nodes can synchronously receive the required data. Then, the written node configuration instructions are used to configure each receiving node separately, and the configuration status (resource utilization) of all receiving nodes is monitored using the existing Zabbix tool. Furthermore, Zabbix is ​​used to plot the optimized configuration curves of all receiving nodes when receiving the required data. These optimized configuration curves are dynamic curves used to monitor the reception status and latency of all receiving nodes when receiving the required data. The optimized configuration curves include the configuration information of all receiving nodes, which includes configuration instructions, and will not be elaborated further here.

[0115] It should be noted that in this application, when the optimized configuration curve reaches the smoothing standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve. That is, the optimized configuration curve can be dynamically adjusted by adjusting the proportion of the synchronization scheduling value and the synchronization state entropy in the configuration instructions. When the optimized configuration curve reaches the smoothing standard, the configuration instructions in the configuration information on the optimized configuration curve control all receiving nodes and assign synchronization priorities to the receiving nodes. Based on the assigned synchronization priorities, the required data is synchronized across multiple receiving nodes. If the optimized configuration curve does not reach the smoothing standard, the proportion of the synchronization scheduling value and the synchronization state entropy in the configuration instructions continues to be adjusted until the optimized configuration curve reaches the smoothing standard. The smoothing standard refers to the slope of the tangent line of the optimized configuration curve being 0. In addition, by adjusting the synchronization priority of all receiving nodes, the synchronization latency of each receiving node can be reduced, which is beneficial to improving the efficiency of concurrent synchronization of required data. The synchronization priority refers to the different processing orders and resource occupancy rates assigned to receiving nodes by the system.

[0116] On the other hand, in some embodiments, this application provides a customer demand management system, which includes a data synchronization unit, referencing... Figure 4 The figure is a schematic diagram of the structure of a data synchronization unit in some embodiments of this application. The data synchronization unit 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below:

[0117] The acquisition module 401 in this application is mainly used to acquire the synchronization record of the demand data and obtain the demand synchronization data.

[0118] Processing module 402, in this application, is used to determine the contention margin of each receiving node when synchronously receiving the demand data based on the configuration information of each receiving node during the synchronization process of the demand data and the allocation log in the demand synchronization data.

[0119] In this application, the processing module 402 is also used to audit the demand data received synchronously by all receiving nodes, obtain the audit amount when the demand data is synchronized, and then determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data based on the audit amount and each contention margin.

[0120] In this application, the processing module 402 is also used to determine the synchronization update status of each receiving node based on the index record of the demand data of each receiving node, and then perform state authentication on the synchronization process of the demand data through each synchronization update status to obtain the synchronization state entropy of all receiving nodes.

[0121] The execution module 403 in this application is mainly used to optimize the configuration information of all receiving nodes according to the synchronization scheduling sequence and the synchronization state entropy, so as to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

[0122] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described data synchronization method for a customer demand management system.

[0123] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a data synchronization method for a customer demand management system, according to some embodiments of this application. The data synchronization method for a customer demand management system in the above embodiments can be implemented through... Figure 5 The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0124] Processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0125] The communication bus 502 can be used to transmit information between the aforementioned components.

[0126] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0127] The memory 503 stores program code for executing the solution of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The data synchronization method for the customer demand management system in the above embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0128] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0129] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0130] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0131] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data synchronization method for a customer demand management system.

[0132] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0133] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data synchronization method for a customer demand management system, characterized in that, Includes the following steps: Obtain the synchronized records of the demand data to get the demand synchronization data; The contention margin of each receiving node when synchronously receiving demand data is determined based on the configuration information of each receiving node during the synchronization process of demand data and the allocation log in the demand synchronization data. Audit the demand data received synchronously by all receiving nodes to obtain the audit quantity when the demand data is synchronized. Then, the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data is determined by the audit quantity and each contention margin. The audit quantity is an indicator that measures the degree of uniformity of demand information received synchronously by different receiving nodes. Based on the index record of each receiving node for the demand data, the synchronization update status of each receiving node is determined, and then the synchronization process of the demand data is authenticated through each synchronization update status to obtain the synchronization state entropy of all receiving nodes, wherein the synchronization state entropy represents the uncertainty measure of the condition state of all receiving nodes when synchronously receiving demand data. Based on the synchronization scheduling sequence and the synchronization state entropy, the configuration information of all receiving nodes is synchronized and optimized to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothness standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

2. The method as described in claim 1, characterized in that, The contention margin for each receiving node during the synchronization of demand data is determined based on the configuration information of each receiving node and the allocation log in the demand synchronization data. Specifically, this includes: Obtain the configuration information of each receiving node during the synchronization process to obtain the required data; The latency characteristics of each receiving node during the required data synchronization process are determined based on the configuration information of each receiving node. Extract the allocation logs of all receiving nodes during the demand data synchronization from the demand data synchronization data; The task allocation list of all receiving nodes during the required data synchronization is determined through the allocation log. The contention margin of each receiving node when synchronously receiving the required data is determined by the task allocation list and the latency characteristics of each receiving node during the required data synchronization process.

3. The method as described in claim 1, characterized in that, Auditing the required data received synchronously by all receiving nodes yields the audited amount specifically including: Determine the attribute entropy of each receiving node; The configuration information of each receiving node is obtained, and then the configuration score of each receiving node is determined based on the configuration information of each receiving node. The audit coefficient of each receiving node is determined by the configuration score and attribute entropy of each receiving node; Obtain the data information stream of each receiving node synchronously receiving the required data; Based on the audit coefficient of each receiving node, the data information flow of each receiving node is audited to obtain the audit quantity when data synchronization is required.

4. The method as described in claim 1, characterized in that, The synchronization scheduling sequence of all receiving nodes when determining the synchronous reception requirement data based on the audit quantity and various contention margins specifically includes: The coordination degree of each receiving node when synchronously receiving required data is determined based on the audit quantity. The process of synchronously receiving data is coordinated and scheduled by the coordination degree and contention margin of each receiving node, so as to obtain the synchronization scheduling value of each receiving node. The synchronization scheduling sequence of all receiving nodes is determined based on the synchronization scheduling value of each receiving node when synchronously receiving the required data.

5. The method as described in claim 1, characterized in that, Determining the synchronization update status of each receiving node based on its index record of the required data specifically includes: Obtain the index record of the required data for each receiving node; Extract the synchronization latency of each receiving node from each index record; Obtain the data information stream of each receiving node synchronously receiving the required data; The synchronization update rate of each receiving node is determined by the synchronization delay and data information flow of each receiving node, thereby determining the synchronization update status of each receiving node.

6. The method as described in claim 1, characterized in that, Based on the synchronization scheduling sequence and the synchronization state entropy, the configuration information of all receiving nodes is synchronized and optimized to obtain the optimized configuration curves of all receiving nodes, specifically including: Obtain the conditional entropy of each receiving node when it synchronously receives the required data; The optimization order of each receiving node is determined based on the conditional entropy of each receiving node and the synchronization state entropy. The configuration information of all receiving nodes is optimized by using the optimization order, the synchronization state entropy, and the synchronization scheduling values ​​in the synchronization scheduling sequence, and the configuration status of all receiving nodes is monitored to obtain the optimized configuration curves of all receiving nodes.

7. The method as described in claim 1, characterized in that, The allocation log is a database log.

8. A customer demand management system, which uses the method described in any one of claims 1 to 7 for data synchronization, the customer demand management system comprising a data synchronization unit, characterized in that, The data synchronization unit includes: The acquisition module is used to acquire the synchronization records of the demand data and obtain the demand synchronization data. The processing module is used to determine the contention margin of each receiving node when synchronously receiving the demand data based on the configuration information of each receiving node during the synchronization process and the allocation log in the demand synchronization data. The processing module is also used to audit the demand data received synchronously by all receiving nodes, obtain the audit amount when the demand data is synchronized, and then determine the synchronization scheduling sequence of all receiving nodes when synchronously receiving demand data based on the audit amount and each contention margin. The processing module is also used to determine the synchronization update status of each receiving node based on the index record of the demand data of each receiving node, and then perform state authentication on the synchronization process of the demand data through each synchronization update status to obtain the synchronization state entropy of all receiving nodes. The execution module is used to perform synchronization optimization on the configuration information of all receiving nodes based on the synchronization scheduling sequence and the synchronization state entropy, to obtain the optimized configuration curve of all receiving nodes. When the optimized configuration curve reaches the smoothing standard, the synchronization priority of each receiving node is adjusted by the configuration information on the optimized configuration curve.

9. A computer device comprising a memory and a processor, the memory storing code, characterized in that, The processor is configured to acquire the code and execute the data synchronization method for a customer demand management system as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the data synchronization method for a customer demand management system as described in any one of claims 1 to 7.

Images