Data processing method and system and related equipment

Abstract

The invention provides a data processing method and system and related equipment, and relates to the technical field of computers, the data processing method is applied to the data processing system.The method comprises the steps that a PDC determines a target weight parameter corresponding to each BDC according to the request frequency and the request type of each BDC, the target weight parameter is used for representing the probability that the BDC expects to store the first type of data, the request frequency represents the frequency that the BDC requests data change from the PDC, and the request type represents the type of the data that the BDC requests data change from the PDC; the PDC determines a first BDC in the plurality of BDCs based on a target weight parameter corresponding to the type of the first data; and the PDC sends a first data change message to the first BDC, wherein the first data change message is used for changing the first data at the first BDC. The dynamic weighted distribution of the data storage task is realized, so that the BDC with a specific data type requirement is preferentially processed, the system resource distribution is optimized, the method can adapt to the rapid increase of the data volume, and the data processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, system, and related equipment. Background Art

[0002] Data backup is a core means of mitigating data loss risks, ensuring business continuity, and ensuring system resilience. It effectively prevents irreversible losses caused by accidents, failures, or attacks. Currently, multi-point data backup typically requires distributing data across multiple physical or virtual locations, such as the cloud or external hard drives. Periodic data copying and synchronization to various backup nodes establishes a data security system and ensures data recovery. However, this approach relies on manual intervention and complex configuration, struggles to adapt to rapid data growth, and results in low data processing efficiency. Summary of the Invention

[0003] The embodiments of the present application provide a data processing method, system and related equipment to solve the problem of low data processing efficiency in the prior art.

[0004] To solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides a data processing method, which is applied to a data processing system, wherein the system includes a central data processing center (PDC) and multiple backup data processing centers (BDCs) in communication with the PDC, wherein the multiple BDCs are deployed at the edge of a communication network; The method comprises: The PDC determines a target weight parameter corresponding to each BDC based on a request frequency and a request type of each BDC, wherein the target weight parameter represents a probability that the BDC expects to store data of a first type, the request frequency represents a frequency at which the BDC requests data changes from the PDC, and the request type represents a type of data for which the BDC requests data changes from the PDC; The PDC determines a first BDC from among the multiple BDCs based on the target weight parameter corresponding to the type of the first data; The PDC sends a first data change message to the first BDC, where the first data change message is used to change the first data in the first BDC.

[0005] Optionally, the PDC determines a target weight parameter corresponding to each BDC according to the request frequency and request type of each BDC, including: The PDC determines a first weight parameter corresponding to each BDC based on a request frequency of each BDC obtained periodically, where the first weight parameter is used to represent a probability that the BDC expects to store data; The PDC determines, based on the request type of each BDC periodically obtained, a second weight parameter corresponding to each BDC, where the second weight parameter is used to represent a probability that the BDC expects to store data associated with the corresponding request type; The PDC obtains the target weight parameter according to the first weight parameter and the second weight parameter.

[0006] Optionally, the PDC determines a second weight parameter corresponding to each BDC according to a request type of each BDC periodically obtained, including: The PDC determines, based on a first number and a second number, a ratio of first requests of the first type in a target BDC, where the first number is the number of times the target BDC requests the PDC for data changes of the first type, and the second number is the total number of times the target BDC requests data changes from the PDC, and the target BDC is any one of the multiple BDCs; The PDC determines, based on a third number and a fourth number, a ratio of second requests of the first type among the multiple BDCs, where the third number is a sum of the number of times the multiple BDCs request data changes of the first type from the PDC, and the fourth number is a sum of the total number of times the multiple BDCs request data changes from the PDC; The PDC determines the relative entropy of the target BDC according to the first request ratio and the second request ratio; The PDC determines a second weight parameter of the target BDC according to the relative entropy of the target BDC.

[0007] Optionally, the PDC sends a first data change message to the first BDC, including: The PDC encapsulates the acquired first data change message into a message event, where the message event is used for transmission in a message queue; The PDC sends the message event to the first BDC based on a message queue corresponding to the first BDC, where each of the multiple BDCs corresponds to a message queue.

[0008] Optionally, the method further includes: When the second BDC receives a data query request sent by the target user, determining the backup status of the second type of data in the second BDC, where the second type is the type of data indicated by the data query request, and the second BDC is the BDC closest to the target user among the multiple BDCs; In a case where the backup state indicates that the second type of data is not stored in the second BDC, the second BDC generates a data backup request, where the data backup request is used to request the second type of data from the PDC; The PDC generates a second data change message according to the data backup request sent by the second BDC, where the second data change message is used for the second type of data backup; The second BDC performs the second type of data backup in the second BDC according to the second data change message sent by the PDC.

[0009] In a second aspect, an embodiment of the present application provides a data processing system, the system comprising a central data processing center (PDC) and a plurality of backup data processing centers (BDCs) in communication with the PDC, the plurality of BDCs being deployed at the edge of a communication network; wherein, The PDC is configured to perform the following operations: Determining a target weight parameter corresponding to each BDC based on a request frequency and a request type of each BDC, wherein the target weight parameter is used to represent a probability that the BDC expects to store data of a first type, the request frequency represents a frequency at which the BDC requests data changes from the PDC, and the request type represents a type of data for which the BDC requests data changes from the PDC; determining a first BDC among the plurality of BDCs based on the target weight parameter corresponding to the type of the first data; A first data change message is sent to the first BDC, where the first data change message is used to change the first data in the first BDC.

[0010] Optionally, the PDC is specifically used for: Determining a first weight parameter corresponding to each BDC based on a request frequency of each BDC obtained periodically, wherein the first weight parameter is used to represent a probability that the BDC expects to store data; Determining, based on the request type of each BDC periodically obtained, a second weight parameter corresponding to each BDC, wherein the second weight parameter is used to represent a probability that the BDC expects to store data associated with the corresponding request type; The target weight parameter is obtained according to the first weight parameter and the second weight parameter.

[0011] Optionally, the PDC is specifically used for: determining, based on a first number and a second number, a first request ratio of the first type in a target BDC, wherein the first number is the number of times the target BDC requests data changes of the first type from the PDC, and the second number is the total number of times the target BDC requests data changes from the PDC, the target BDC being any one of the multiple BDCs; determining a ratio of second requests of the first type among the multiple BDCs based on a third number and a fourth number, wherein the third number is a sum of the number of times the multiple BDCs request data changes of the first type from the PDC, and the fourth number is a sum of the total number of times the multiple BDCs request data changes from the PDC; determining a relative entropy of the target BDC according to the first request ratio and the second request ratio; A second weight parameter of the target BDC is determined according to the relative entropy of the target BDC.

[0012] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor, a memory, and a program stored on the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the data processing method as described in the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data processing method described in the first aspect are implemented.

[0014] In this embodiment, the PDC dynamically analyzes the BDC's data request frequency and request type, determines a target weight parameter representing the probability of data storage adaptation, and intelligently matches the optimal BDC for a specific data type based on the target weight parameter to send a change message. This achieves dynamic weighted allocation of data storage tasks, prioritizing BDCs with specific data type requirements and optimizing system resource allocation. Compared to methods that rely on manual intervention, intelligent message allocation ensures real-time and accurate synchronization of data changes, adapts to rapid data growth, and improves data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1This is one of the flow charts of a data processing method provided in an embodiment of the present application; Figure 2 This is a structural diagram of a data processing system provided in an embodiment of the present application; Figure 3 This is the second flowchart of a data processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0018] The terms "first," "second," and the like in the embodiments of the present application are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to the process, method, product, or apparatus.

[0019] See also Figure 1 and Figure 2 , Figure 1 This is one of the flow charts of a data processing method provided in an embodiment of the present application. Figure 2 The data processing method provided in the embodiment of the present application can be applied to a data processing system, which can be a distributed system architecture consisting of a primary data processing center (PDC) and multiple backup data processing centers (BDCs) in communication with the PDC. The multiple BDCs are deployed at the edge of a communication network to reduce data transmission delays and improve response speeds. The BDCs can also serve as edge service nodes for serving users physically close to the edge service nodes. These nodes are connected via a network and exchange data via message queues.

[0020] The method comprises the following steps: Step 101: The PDC determines a target weight parameter corresponding to each BDC based on the request frequency and request type of each BDC, where the target weight parameter represents the probability that the BDC expects to store data of a first type, the request frequency represents the frequency with which the BDC requests data changes from the PDC, and the request type represents the type of data for which the BDC requests data changes from the PDC. In this step, the PDC dynamically calculates a target weight parameter for each BDC by analyzing each BDC's data request behavior, combining the frequency with which each BDC requests data changes (i.e., request frequency) and the specific type of data change requested (i.e., request type). The target weight parameter quantifies the BDC's preference for a specific type (e.g., type 1). A higher target weight parameter assigned to a BDC increases the priority and expected probability of processing type 1 data for that BDC, thus providing a data foundation for subsequent intelligent allocation.

[0021] Step 102: The PDC determines a first BDC from among the multiple BDCs based on the target weight parameter corresponding to the type of the first data. In this step, the first data may be data in the PDC that needs to be backed up to the BDC, and the type of the first data may be any of the types such as bills, subscription relationships, and user provinces. The target weight parameter corresponding to the type of the first data for each PDC may be different. The higher the value of the target weight parameter, the more inclined the BDC corresponding to the target weight parameter is to process the first data. In other words, the higher the target weight parameter, the more likely the BDC is to be selected as the first BDC in a probabilistic sense. In this way, based on the target weight parameter corresponding to the type of the first data, the first BDC is determined among multiple BDCs to ensure that the node whose type of the first data better matches the historical request characteristics of the first BDC takes on the data change task with a higher probability, thereby optimizing the adaptability of data storage and the overall system efficiency.

[0022] Step 103: The PDC sends a first data change message to the first BDC, where the first data change message is used to change the first data in the first BDC.

[0023] In this step, after determining the first BDC to carry the data change, the PDC sends a first data change message containing complete change information to the first BDC. This message carries the key instructions required to replicate the data change on the BDC (such as add, delete, and modify operations, data mirroring, etc.). It drives the first BDC to perform the corresponding write operations based on the received change information, thereby achieving data synchronization between the PDC and BDC, ensuring that the backup data processing center can accurately reflect data status changes in the central data processing center in real time.

[0024] In this embodiment, the PDC dynamically analyzes the BDC's data request frequency and request type, determines a target weight parameter representing the probability of data storage adaptation, and intelligently matches the optimal BDC for a specific data type based on the target weight parameter to send a change message. This achieves dynamic weighted allocation of data storage tasks, prioritizing BDCs with specific data type requirements and optimizing system resource allocation. Compared to methods that rely on manual intervention, intelligent message allocation ensures real-time and accurate synchronization of data changes, adapts to rapid data growth, and improves data processing efficiency.

[0025] Optionally, the PDC determines a target weight parameter corresponding to each BDC according to the request frequency and request type of each BDC, including: The PDC determines a first weight parameter corresponding to each BDC based on a request frequency of each BDC obtained periodically, where the first weight parameter is used to represent a probability that the BDC expects to store data; The PDC determines, based on the request type of each BDC periodically obtained, a second weight parameter corresponding to each BDC, where the second weight parameter is used to represent a probability that the BDC expects to store data associated with the corresponding request type; The PDC obtains the target weight parameter according to the first weight parameter and the second weight parameter.

[0026] In this embodiment, the PDC periodically collects the data request frequency (i.e., the frequency of data change requests) and request type (i.e., the data type of the specific request) of each BDC, and generates a first weight parameter (based on frequency) that represents the overall data storage adaptation probability of the BDC and a second weight parameter (based on type) that represents the data storage adaptation probability of a specific request type. By integrating the weight parameters of these two dimensions, a target weight parameter is finally formed that comprehensively reflects the BDC's expected probability of a specific type of data storage, providing a quantitative basis for the dynamic and intelligent allocation of data storage tasks.

[0027] Specifically, in the initial stage, a basic weight parameter is set for each BDC. Initially, the basic weight parameter of all BDCs can be the same (for example, if there are N BDCs, the basic weight parameter of each BDC is 1 / N). In addition, a global request counter is maintained for each data type to record the total number of times that type of data is requested by all BDCs.

[0028] During runtime, request frequencies are updated in real time, with a request frequency counter maintained for each BDC and each data type. Each time a BDC requests data, the request frequency counter for that BDC and data type is updated. Furthermore, by segmenting data types by geographic region or business type and analyzing their request frequencies, insights can be gained into the data needs of different regions or business lines, providing strong support for more precise market strategy and business planning. Based on real-time updates of request frequencies, BDC layout and configuration can be continuously optimized, improving data processing efficiency and service quality, further enhancing market competitiveness.

[0029] In one example, the first weight parameter corresponding to each BDC is determined as follows: The PDC periodically (e.g., hourly or daily) calculates a new first weight parameter based on the BDC's request frequency. The first weight parameter can be proportional to the request frequency, for example, using an exponential or logarithmic function of the number of requests as the basis for calculating the first weight parameter to avoid weight skew in extreme cases. The formula for calculating the first weight parameter can be expressed as: New first weight parameter = old first weight parameter * (1 + α * (number of requests - average number of requests) / average number of requests); Among them, the old first weight parameter in the initial state is 1 / N, α is an adjustment factor used to control the sensitivity of weight adjustment, the number of requests represents the total number of times BDC requests data changes from PDC, and the average number of requests represents the average of the total number of times BDC requests data changes from PDC.

[0030] In addition, to ensure that the sum of the first weight parameters corresponding to each BDC is 1, normalization processing can be performed to facilitate subsequent probability allocation.

[0031] Furthermore, considering the data type tendency, the type of BDC's request data (i.e., request type) can be regularly analyzed to identify whether each BDC has a clear request type tendency. The PDC determines the second weight parameter corresponding to each BDC based on the request type of each BDC obtained periodically, and finally forms a target weight parameter that comprehensively reflects the BDC's expected probability of storing a specific type of data by fusing the first weight parameter and the second weight parameter, providing a quantitative basis for the dynamic and intelligent allocation of data storage tasks.

[0032] In some optional embodiments, the second weight parameter corresponding to each BDC may be determined by comparing the difference between the data type distribution requested by the BDC and the global data type distribution, as described in the following: The PDC determines, based on a first number and a second number, a ratio of first requests of the first type in a target BDC, where the first number is the number of times the target BDC requests the PDC for data changes of the first type, and the second number is the total number of times the target BDC requests data changes from the PDC, and the target BDC is any one of the multiple BDCs; The PDC determines, based on a third number and a fourth number, a ratio of second requests of the first type among the multiple BDCs, where the third number is a sum of the number of times the multiple BDCs request data changes of the first type from the PDC, and the fourth number is a sum of the total number of times the multiple BDCs request data changes from the PDC; The PDC determines the relative entropy of the target BDC according to the first request ratio and the second request ratio; The PDC determines a second weight parameter of the target BDC according to the relative entropy of the target BDC.

[0033] In this embodiment, the usage of each data type across the entire system is first collected and counted, including the number of requests and data size, to determine the global data type distribution. This data can be obtained from system logs, database query records, or monitoring tools. Furthermore, the data type and request frequency of each BDC request are collected, for example, from logs or statistical information reported by the BDC when requesting data.

[0034] Then, the collected data will be organized to ensure that the statistical caliber of each data type is consistent at the global and BDC levels; a unified naming rule will be established for the data type to ensure the consistency and recognizability of the name; and the same time period will be selected for comparison to eliminate errors caused by time asynchrony or data delays.

[0035] Afterwards, the first request ratio of the first type in the target BDC is determined based on the first quantity and the second quantity. The first quantity may be the number of times the target BDC requests the PDC for data changes of the first type, and the second quantity may be the total number of times the target BDC requests the PDC for data changes. In other words, the first request ratio may be the BDC data type ratio. For each BDC (taking the target BDC as an example), the ratio of each data type (taking the first type as an example) in its request may be calculated: the first request ratio = (first quantity / second quantity), that is, the ratio of the first type of data in the target BDC request = (number of BDC requests for this data type / total number of BDC requests).

[0036] Based on the third quantity and the fourth quantity, the proportion of second requests of the first type in multiple BDCs is determined. The third quantity may be the sum of the number of times multiple BDCs request the PDC for data changes of the first type, and the fourth quantity may be the sum of the total number of times multiple BDCs request the PDC for data changes. In other words, the second request proportion may be the global data type proportion. For each data type (taking the first type as an example), the proportion of each data type in the global request is calculated: the second request proportion = (third quantity / fourth quantity), that is, the proportion of the first type of data in the global request = (number of global requests for the data type / total number of global requests).

[0037] After calculating the first and second request ratios, the relative entropy of the target BDC is determined based on the first and second request ratios. Calculating the relative entropy uses the Kullback-Leibler Divergence (KL Divergence). KL Divergence measures the difference between two probability distributions, P and Q. In this example, it can be used to measure the difference between the BDC data type request distribution and the global data type distribution.

[0038] In one example, let the global data type distribution be P, where P(i) represents the global request ratio of the i-th data type (i.e., the second request ratio). Let the data type request distribution of a BDC be Q, where Q(i) represents the request ratio of the i-th data type in the BDC (i.e., the first request ratio). Calculate the KL divergence: ; Among them, D kl is the KL divergence value. Smaller KL divergence values ​​indicate closer distributions; larger values ​​indicate greater differences. For each BDC, its KL divergence with the global data type distribution can be calculated to quantify the difference in its data type request.

[0039] The PDC determines a second weight parameter for each of the multiple BDCs based on their relative entropy. This second weight parameter setting allows for preferential allocation. For example, if a BDC is found to have a significant preference for a particular type of data, an additional second weight parameter for that type of data is added to the BDC's first weight parameter. The size of this additional weight can be determined based on the significance of the preference and the importance of the data type.

[0040] During each backup operation, the PDC randomly selects or allocates data proportionally based on the BDC's current target weight parameter. For example, a roulette wheel selection algorithm can be used to select BDCs based on weight. Based on the target weight parameter corresponding to the first data type, after determining the first BDC among multiple BDCs, the preferred data type is preferentially sent to it. If no preferred data type exists, data is randomly or proportionally allocated based on the global data type distribution. By integrating the first and second weight parameters, the data type preferences of BDCs are taken into account, prioritizing the BDCs that require specific data types. This targeted processing not only reduces unnecessary data transmission but also shortens data request response time, improving data processing efficiency. Furthermore, based on the BDC's request frequency and data type preferences, the BDC's target weight parameter can be calculated and adjusted in real time. This dynamic weight adjustment mechanism ensures that backup tasks can be flexibly allocated according to actual needs, reducing resource waste or concentrated backup pressure caused by static allocation, thereby further improving overall backup efficiency.

[0041] Optionally, the PDC sends a first data change message to the first BDC, including: The PDC encapsulates the acquired first data change message into a message event, where the message event is used for transmission in a message queue; The PDC sends the message event to the first BDC based on a message queue corresponding to the first BDC, where each of the multiple BDCs corresponds to a message queue.

[0042] In this embodiment, the PDC can record data changes in a change log (such as a binlog) through an associated database. A change log capture service (which can be a standalone service or part of a database) monitors and captures new change log entries in real time. The PDC encapsulates the captured change log entries (i.e., first data change messages) into message events. Each message event contains sufficient information to replay the corresponding data change on the BDC. A message queue (such as Kafka or RabbitMQ) is used as the message event middleware. The change log capture service publishes the encapsulated message events to multiple corresponding message queues (one for each BDC) based on a message event distribution algorithm, where they await consumption. The data synchronization service on the BDC acts as a consumer of the message queue and subscribes to its own message queue. When new message events arrive, the data synchronization service asynchronously pulls these messages and processes them. For each message event, the data synchronization service parses the change instructions contained therein and performs the corresponding write operation on the BDC's database to replicate the data change on the PDC. Upon completion, the data synchronization service sends a confirmation message to the message queue, indicating that the message event has been successfully processed.

[0043] Change log capture and encapsulation monitors and captures database change log entries in real time, encapsulating them into message events. This process enables instant capture of data changes, avoiding potential data omissions in traditional backups and ensuring the integrity and accuracy of backup data. Furthermore, encapsulation into message events facilitates efficient subsequent processing and transmission, reducing processing delays and improving backup efficiency.

[0044] Furthermore, using message queues as intermediaries, encapsulated message events are distributed to multiple corresponding message queues according to a distribution algorithm. This approach enables parallel processing of backup tasks, allowing multiple BDCs to simultaneously receive and process backup data, significantly improving the concurrency and processing speed of backup operations.

[0045] Furthermore, the data synchronization service on the BDC asynchronously pulls and processes event messages, performing corresponding write operations to replicate data changes on the PDC. This asynchronous processing mechanism reduces the impact of backup operations on the main system processes, ensuring high system availability. Furthermore, precise replication of change instructions ensures consistency between backup data and the original data, improving backup accuracy.

[0046] like Figure 3 As shown, the data processing method provided in this embodiment may further include the following steps: Step 301, PDC setup and initialization: The PDC sets and initializes the message backup message distribution algorithm, and can determine the target weight parameter through the distribution algorithm. The specific process can be found in the above content and will not be repeated here. The algorithm is regularly updated to adapt to system changes.

[0047] Step 302, backup process: The PDC distributes the newly generated message to the corresponding BDC (edge ​​service node) for backup according to the current message distribution algorithm.

[0048] Step 303: BDC saves the message: After obtaining the corresponding message, BDC saves it in the local database.

[0049] Step 304: The BDC fulfills the user data request: The BDC also functions as an edge router. When a user sends a data request to the corresponding edge router (i.e., the BDC), the BDC first checks whether it has the requested data stored locally. If the BDC has the requested data stored locally, it directly provides the data to the user. If the BDC does not have the requested data stored locally, it initiates a request to the PDC, which then provides the corresponding data. For details, see the following description: Optionally, the method further includes: When the second BDC receives a data query request sent by the target user, determining the backup status of the second type of data in the second BDC, where the second type is the type of data indicated by the data query request, and the second BDC is the BDC closest to the target user among the multiple BDCs; In a case where the backup state indicates that the second type of data is not stored in the second BDC, the second BDC generates a data backup request, where the data backup request is used to request the second type of data from the PDC; The PDC generates a second data change message according to the data backup request sent by the second BDC, where the second data change message is used for the second type of data backup; The second BDC performs the second type of data backup in the second BDC according to the second data change message sent by the PDC.

[0050] In this embodiment, the system first routes the user's request to the edge node (i.e., the second BDC) closest to the target user's physical location based on geographic proximity to minimize network latency. The second BDC then quickly checks its own storage status, using metadata indexes to determine whether the requested second type of data is already cached. If the check indicates that the requested data type is not already stored, the second BDC immediately initiates an asynchronous data backup request to the PDC. This request can include information such as a data type identifier, timestamp, and user location to precisely describe the characteristics of the requested data.

[0051] After receiving the request, the PDC invokes the data change generation module, which constructs a second data change message based on the global data view. This message includes incremental backup instructions, data verification mechanisms, and point-in-time consistency guarantees. The generated change message is transmitted to the second BDC through its corresponding message queue.

[0052] After receiving the change message, the second BDC initiates the backup process. Once the backup is complete, the second BDC returns the query results to the user and incorporates the newly backed-up data into its local caching strategy, allowing subsequent similar requests to be directly responded to at the edge node. In this way, the BDC not only serves as a storage center for message backups but also integrates the functionality of an edge router, enabling localized data processing and rapid response. This design simplifies the system architecture, reduces maintenance costs, and speeds up the processing of user requests, enabling users to access the data they need more quickly.

[0053] The embodiment of the present application further provides a data processing system, the system comprising a central data processing center (PDC) and a plurality of backup data processing centers (BDCs) in communication with the PDC, the plurality of BDCs being deployed at the edge of a communication network; wherein, The PDC is configured to perform the following operations: Determining a target weight parameter corresponding to each BDC based on a request frequency and a request type of each BDC, wherein the target weight parameter is used to represent a probability that the BDC expects to store data of a first type, the request frequency represents a frequency at which the BDC requests data changes from the PDC, and the request type represents a type of data for which the BDC requests data changes from the PDC; determining a first BDC among the plurality of BDCs based on the target weight parameter corresponding to the type of the first data; A first data change message is sent to the first BDC, where the first data change message is used to change the first data in the first BDC.

[0054] Optionally, the PDC is specifically used for: Determining a first weight parameter corresponding to each BDC based on a request frequency of each BDC obtained periodically, wherein the first weight parameter is used to represent a probability that the BDC expects to store data; Determining, based on the request type of each BDC periodically obtained, a second weight parameter corresponding to each BDC, wherein the second weight parameter is used to represent a probability that the BDC expects to store data associated with the corresponding request type; The target weight parameter is obtained according to the first weight parameter and the second weight parameter.

[0055] Optionally, the PDC is specifically used for: determining, based on a first number and a second number, a first request ratio of the first type in a target BDC, wherein the first number is the number of times the target BDC requests data changes of the first type from the PDC, and the second number is the total number of times the target BDC requests data changes from the PDC, the target BDC being any one of the multiple BDCs; determining a ratio of second requests of the first type among the multiple BDCs based on a third number and a fourth number, wherein the third number is a sum of the number of times the multiple BDCs request data changes of the first type from the PDC, and the fourth number is a sum of the total number of times the multiple BDCs request data changes from the PDC; determining a relative entropy of the target BDC according to the first request ratio and the second request ratio; A second weight parameter of the target BDC is determined according to the relative entropy of the target BDC.

[0056] Optionally, the PDC is specifically used for: Encapsulating the acquired first data change message into a message event, where the message event is used for transmission in a message queue; The message event is sent to the first BDC based on a message queue corresponding to the first BDC, where each of the multiple BDCs corresponds to a message queue.

[0057] Optionally, the second BDC among the multiple BDCs is configured to perform the following operations: When the second BDC receives a data query request sent by the target user, determining the backup status of the second type of data in the second BDC, where the second type is the type of data indicated by the data query request, and the second BDC is the BDC closest to the target user among the multiple BDCs; In a case where the backup state indicates that the second type of data is not stored in the second BDC, the second BDC generates a data backup request, where the data backup request is used to request the second type of data from the PDC; The PDC is also used to perform the following operations: generating a second data change message according to the data backup request sent by the second BDC, where the second data change message is used for the second type of data backup; The second BDC is also used to perform the following operations: The second type of data backup is performed in the second BDC according to the second data change message sent by the PDC.

[0058] The data processing system is capable of implementing each process of each embodiment of the above-mentioned data processing method. The technical features correspond one to one and can achieve the same technical effect. To avoid repetition, they will not be described here.

[0059] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a program stored in the memory and runnable on the processor. When the program is executed by the processor, each process of the above-mentioned data processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0060] The present application also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the various processes of the above-described data processing method embodiment and achieves the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0061] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned data processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0062] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing the functions in the order discussed, but may also include performing the functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0063] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0064] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A data processing method, characterized in that: Applied to a data processing system, the system comprises a central data processing center (PDC) and a plurality of backup data processing centers (BDCs) in communication connection with the PDC, the plurality of BDCs being deployed at the edge of a communication network; The method comprises: The PDC determines a target weight parameter corresponding to each BDC based on a request frequency and a request type of each BDC, wherein the target weight parameter represents a probability that the BDC expects to store data of a first type, the request frequency represents a frequency at which the BDC requests data changes from the PDC, and the request type represents a type of data for which the BDC requests data changes from the PDC; The PDC determines a first BDC from among the multiple BDCs based on the target weight parameter corresponding to the type of the first data; The PDC sends a first data change message to the first BDC, where the first data change message is used to change the first data in the first BDC.

2. The method according to claim 1, characterized in that The PDC determines the target weight parameter corresponding to each BDC according to the request frequency and request type of each BDC, including: The PDC determines a first weight parameter corresponding to each BDC based on a request frequency of each BDC obtained periodically, where the first weight parameter is used to represent a probability that the BDC expects to store data; The PDC determines, based on the request type of each BDC periodically obtained, a second weight parameter corresponding to each BDC, where the second weight parameter is used to represent a probability that the BDC expects to store data associated with the corresponding request type; The PDC obtains the target weight parameter according to the first weight parameter and the second weight parameter.

3. The method according to claim 2, characterized in that The PDC determines a second weight parameter corresponding to each BDC according to a request type of each BDC periodically obtained, including: The PDC determines, based on a first number and a second number, a ratio of first requests of the first type in a target BDC, where the first number is the number of times the target BDC requests the PDC for data changes of the first type, and the second number is the total number of times the target BDC requests data changes from the PDC, and the target BDC is any one of the multiple BDCs; The PDC determines, based on a third number and a fourth number, a ratio of second requests of the first type among the multiple BDCs, where the third number is a sum of the number of times the multiple BDCs request data changes of the first type from the PDC, and the fourth number is a sum of the total number of times the multiple BDCs request data changes from the PDC; The PDC determines the relative entropy of the target BDC according to the first request ratio and the second request ratio; The PDC determines a second weight parameter of the target BDC according to the relative entropy of the target BDC.

4. The method according to claim 1, wherein The PDC sends a first data change message to the first BDC, including: The PDC encapsulates the acquired first data change message into a message event, where the message event is used for transmission in a message queue; The PDC sends the message event to the first BDC based on a message queue corresponding to the first BDC, where each of the multiple BDCs corresponds to a message queue.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: When the second BDC receives a data query request sent by the target user, determining the backup status of the second type of data in the second BDC, where the second type is the type of data indicated by the data query request, and the second BDC is the BDC closest to the target user among the multiple BDCs; In a case where the backup state indicates that the second type of data is not stored in the second BDC, the second BDC generates a data backup request, where the data backup request is used to request the second type of data from the PDC; The PDC generates a second data change message according to the data backup request sent by the second BDC, where the second data change message is used for the second type of data backup; The second BDC performs the second type of data backup in the second BDC according to the second data change message sent by the PDC.

6. A data processing system, characterized in that: The system includes a central data processing center PDC and multiple backup data processing centers BDC in communication connection with the PDC, wherein the multiple BDCs are deployed at the edge of the communication network; wherein, The PDC is configured to perform the following operations: Determining a target weight parameter corresponding to each BDC based on a request frequency and a request type of each BDC, wherein the target weight parameter is used to represent a probability that the BDC expects to store data of a first type, the request frequency represents a frequency at which the BDC requests data changes from the PDC, and the request type represents a type of data for which the BDC requests data changes from the PDC; determining a first BDC among the plurality of BDCs based on the target weight parameter corresponding to the type of the first data; A first data change message is sent to the first BDC, where the first data change message is used to change the first data in the first BDC.

7. The system according to claim 6, characterized in that The PDC is specifically used for: Determining a first weight parameter corresponding to each BDC based on a request frequency of each BDC obtained periodically, wherein the first weight parameter is used to represent a probability that the BDC expects to store data; Determining, based on the request type of each BDC periodically obtained, a second weight parameter corresponding to each BDC, wherein the second weight parameter is used to represent a probability that the BDC expects to store data associated with the corresponding request type; The target weight parameter is obtained according to the first weight parameter and the second weight parameter.

8. The system according to claim 7, characterized in that The PDC is specifically used for: determining, based on a first number and a second number, a first request ratio of the first type in a target BDC, wherein the first number is the number of times the target BDC requests data changes of the first type from the PDC, and the second number is the total number of times the target BDC requests data changes from the PDC, the target BDC being any one of the multiple BDCs; determining a ratio of second requests of the first type among the multiple BDCs based on a third number and a fourth number, wherein the third number is a sum of the number of times the multiple BDCs request data changes of the first type from the PDC, and the fourth number is a sum of the total number of times the multiple BDCs request data changes from the PDC; determining a relative entropy of the target BDC according to the first request ratio and the second request ratio; A second weight parameter of the target BDC is determined according to the relative entropy of the target BDC.

9. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the data processing method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 5.

Images