A data processing method, apparatus, device, and storage medium

Abstract

This disclosure provides a data processing method, apparatus, device, and storage medium, comprising: a global management node receiving a parameter change request, updating global parameters according to the parameter change request, generating a parameter change log, and storing it in a parameter change log table; based on the parameter change log, the global management node generating a parameter synchronization task and recording the parameter synchronization task in a parameter synchronization task table; according to the parameter synchronization task, the global management node distributing the parameter change data to target nodes; the target nodes including microservice management nodes or distributed control nodes; and the target nodes updating the parameter change data to their local parameter tables. This solution ensures parameter consistency and avoids business conflicts through centralized version management; it adopts a differentiated distribution strategy to balance the synchronization efficiency of centralized and distributed nodes, achieving unified latency; and by combining log recording and standardized task scheduling, it makes parameter changes traceable, improves troubleshooting efficiency, and significantly enhances system stability.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of technology, the complexity of core business systems is constantly increasing, and their architecture is gradually evolving towards microservices. The fourth generation of the core business system includes multiple types of microservice management nodes and supports a hybrid centralized and distributed deployment. During system operation, each microservice needs to share a large number of public parameters, while some microservices also need to manage their own unique local parameters. In existing technologies, parameter synchronization mainly faces the following challenges:

[0003] First, there is insufficient compatibility in hybrid deployment environments. The node structures of centralized microservices and distributed microservices differ greatly, making it difficult to uniformly adapt traditional synchronization methods. This often results in problems such as low synchronization efficiency of centralized nodes and high synchronization latency of distributed nodes.

[0004] Secondly, there is a lack of version management mechanisms. The various microservice management nodes lack a unified version identifier, making it impossible to quickly verify version consistency after parameter changes. This can easily lead to some nodes using older parameters, causing business logic conflicts.

[0005] Third, the traceability and controllability of the synchronization process are insufficient. Parameter change operations are not fully recorded, and the generation and execution of synchronization tasks lack standardized procedures, making it difficult to locate specific fault nodes or operation steps during problem troubleshooting. Summary of the Invention

[0006] This disclosure provides a data processing method, apparatus, device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0007] According to a first aspect of this disclosure, a data processing method is provided, the method comprising:

[0008] The global management node receives parameter change requests, updates global parameters according to the parameter change requests, generates parameter change logs, and stores them in the parameter change log table;

[0009] Based on the parameter change log, the global management node generates a parameter synchronization task and records the parameter synchronization task in the parameter synchronization task table;

[0010] According to the parameter synchronization task, the global management node distributes the parameter change data to the target node; the target node includes a microservice management node or a distributed control node, and the parameter change data includes a parameter change log table and a global version number.

[0011] The target node updates the parameter change data to its local parameter table.

[0012] In one possible implementation, the step of generating a parameter synchronization task based on the parameter change log and recording the parameter synchronization task in a parameter synchronization task table includes:

[0013] The global management node polls the parameter change log table to determine whether there are any pending parameter change log records.

[0014] When there is a pending parameter change log record, the global management node generates a single synchronization task for the specified number of parameters and records the parameter synchronization task in the parameter synchronization task table.

[0015] In one possible implementation, the step of synchronizing the parameter task by the management node distributing parameter change data to the target node includes:

[0016] The global management node sends the parameter change data to the microservice system configured in the parameter synchronization node information table;

[0017] If the microservice system is deployed in a distributed manner, the parameter change data is sent from the global management node to the microservice management node, and the microservice management node further distributes the parameter change data to its subordinate distributed control nodes;

[0018] If the microservice system is deployed in a centralized manner, the parameter change data is directly sent from the global management node to the microservice management node.

[0019] In one possible implementation, after the target node updates and synchronizes the parameter change data to the local parameter table, when the parameter is globally cached, the method further includes:

[0020] The target node generates a new parameter version number and updates the mapping relationship between the global version number and the local version number.

[0021] After the target node updates the parameter change data to its local parameter table, all target nodes load the parameters into memory and switch versions according to the mapping relationship.

[0022] In one possible implementation, all target nodes load parameters into memory, including:

[0023] The target node polls to detect the version upgrade status of its parameters;

[0024] The instance that wins the lock loads the latest parameters into memory and updates the status in the instance parameter version information table;

[0025] Instances that fail to acquire the lock obtain the latest version number based on the version upgrade status parameters and complete the memory loading.

[0026] In one possible implementation, the version switching based on the mapping relationship includes:

[0027] After all target nodes have been successfully loaded, the parent management node of the target nodes issues a version switching instruction;

[0028] After the superior management node confirms that all target nodes have been successfully switched, it updates the status of the parameter synchronization log table and the parameter synchronization task table.

[0029] In one possible implementation, the parent management node of the target node issues a version switching instruction, including:

[0030] If the microservice system is deployed in a distributed manner, the microservice management node issues a version switching instruction to the distributed control node, and the distributed control node reports the completion status to the microservice management node after completing the switch.

[0031] If the microservice system is deployed in a centralized manner, the microservice management node directly performs version switching and reports the completion status to the global management node.

[0032] According to a second aspect of this disclosure, a data processing apparatus is provided, the apparatus comprising:

[0033] The first generation module is used for the global management node to receive parameter change requests, update global parameters according to the parameter change requests, generate parameter change logs and store them in the parameter change log table;

[0034] The second generation module is used to generate parameter synchronization tasks by the global management node based on the parameter change log, and record the parameter synchronization tasks in the parameter synchronization task table;

[0035] The distribution module is used to synchronize tasks according to the parameters. The global management node distributes the parameter change data to the target node. The target node includes a microservice management node or a distributed control node. The parameter change data includes a parameter change log table and a global version number.

[0036] The update module is used by the target node to update the parameter change data to the local parameter table.

[0037] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0038] At least one processor; and

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0041] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0042] The data processing method, apparatus, equipment, and storage medium disclosed herein receive parameter change requests through a global management node, update global parameters according to the requests, generate parameter change logs, and store them in a parameter change log table. Based on the parameter change logs, the global management node generates parameter synchronization tasks and records these tasks in a parameter synchronization task table. According to the parameter synchronization tasks, the global management node distributes the parameter change data to target nodes, including microservice management nodes or distributed control nodes. The target nodes update their local parameter tables with the parameter change data. This solution ensures parameter consistency and avoids business conflicts through centralized version management; it employs a differentiated distribution strategy to balance the synchronization efficiency of centralized and distributed nodes, achieving unified latency; and it combines log recording and standardized task scheduling to make parameter changes traceable, improve troubleshooting efficiency, and significantly enhance system stability.

[0043] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0044] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0045] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0046] Figure 1 This illustration shows the implementation flow of the data processing method according to an embodiment of the present disclosure. Figure 1 ;

[0047] Figure 2 This illustration shows the implementation flow of the data processing method according to an embodiment of the present disclosure. Figure 2 ;

[0048] Figure 3 This illustration shows the implementation flow of the data processing method according to an embodiment of the present disclosure. Figure 3 ;

[0049] Figure 4 This illustration shows the implementation flow of the data processing method according to an embodiment of the present disclosure. Figure 4 ;

[0050] Figure 5 This illustration shows the implementation flow of the data processing method according to an embodiment of the present disclosure. Figure 5 ;

[0051] Figure 6 This illustration shows the implementation flow of the data processing method according to an embodiment of the present disclosure. Figure 6 ;

[0052] Figure 7 A schematic diagram of the structure of a data processing apparatus according to an embodiment of the present disclosure is shown;

[0053] Figure 8 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0054] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0055] This disclosure provides a data processing method, such as Figure 1 As shown, the method includes:

[0056] Step 101: The global management node receives the parameter change request, updates the global parameters according to the parameter change request, generates a parameter change log, and stores it in the parameter change log table.

[0057] In this example, the global management node is the only node in the entire system responsible for managing the global parameters shared by all microservice systems (including centralized and distributed microservice systems). Parameter change requests originate from the visual management system, such as web-based counter operations. When the global management node receives a parameter change request and performs CRUD operations on the parameter table, it generates a parameter change log. The parameter change log specifically records the specific changes made to the predefined parameter table, such as adding or modifying certain parameter values, and is subsequently stored in the parameter change log table. By recording changes, accurate data is provided for subsequent synchronization, ensuring that each node executes the same change operations, thereby guaranteeing traceability of parameter changes and avoiding synchronization deviations.

[0058] Step 102: Based on the parameter change log, the global management node generates a parameter synchronization task and records the parameter synchronization task in the parameter synchronization task table.

[0059] In this example, the global management node polls the parameter change log table and generates parameter synchronization tasks. These tasks are categorized into global cache and non-global cache tasks, which are processed alternately. On the first system startup, non-global cache tasks are processed first. The generated tasks are recorded in the parameter synchronization task table to track synchronization progress. By transforming the change log into ordered tasks, batch operations are avoided from putting pressure on the system or causing data corruption, thus improving synchronization efficiency and rationally prioritizing the processing of different parameter types.

[0060] Step 103: According to the parameter synchronization task, the global management node distributes the parameter change data to the target node; the target node includes a microservice management node or a distributed control node, and the parameter change data includes a parameter change log table and a global version number.

[0061] In this example, the target node is determined by the microservice system deployment method, and is categorized as a microservice management node or a distributed control node depending on whether the microservice system is centralized or distributed. Parameter change data includes a parameter change log table recording specific changes and a global version number. The global version number is used for version verification across multiple systems; its initial value comes from a specific configuration and increments after each global synchronization. The global management node sends data to the target node based on the node list configured in the parameter synchronization node information table. Through differentiated distribution across different deployment environments and version number binding, it ensures that each node receives the correct change data, is compatible with different deployment scenarios, and prevents inconsistencies in node parameter versions.

[0062] Step 104: The target node updates the parameter change data to the local parameter table.

[0063] In this example, the target node receives parameter change data and replays the add, delete, and modify operations in its local parameter table according to the change log sequence. For example, it writes the new parameter value to the local database. Sequential replay ensures the correct execution of the change operations, and version mapping establishes a connection between the global and local tables, providing a foundation for subsequent operations and ensuring consistency between the local parameter table and the global changes, thus avoiding data conflicts.

[0064] This disclosure proposes a data processing method. A global management node receives parameter change requests, updates global parameters based on these requests, generates parameter change logs, and stores them in a parameter change log table. Based on these logs, the global management node generates parameter synchronization tasks and records these tasks in a parameter synchronization task table. According to these tasks, the global management node distributes the parameter change data to target nodes, including microservice management nodes or distributed control nodes. The target nodes then update their local parameter tables with the changed data. This solution ensures parameter consistency and avoids business conflicts through centralized version management; it employs a differentiated distribution strategy to balance the synchronization efficiency of centralized and distributed nodes, achieving unified latency; and it combines log recording and standardized task scheduling to make parameter changes traceable, improve troubleshooting efficiency, and significantly enhance system stability.

[0065] In one example, based on the parameter change log, the global management node generates a parameter synchronization task and records the parameter synchronization task in the parameter synchronization task table, such as... Figure 2 As shown, it includes:

[0066] Step 201: The global management node polls the parameter change log table to determine whether there are any pending parameter change log records.

[0067] In this example, the global management node periodically or continuously checks the parameter change log table through a polling mechanism. The core purpose is to proactively identify parameter change operations that require synchronization. The parameter change log table stores all parameter change records for which database add, delete, and modify operations have been completed but for which synchronization tasks have not yet been generated. These records are marked as "pending" (e.g., adding a new interest rate rule, modifying a fee percentage, etc.). The polling mechanism ensures that change operations are captured promptly, avoiding delays in subsequent synchronization processes due to delayed processing. By proactively detecting changes, trigger conditions are provided for generating synchronization tasks, ensuring the timeliness of parameter synchronization.

[0068] Step 202: When there is a pending parameter change log record, the global management node generates a single synchronization task for the number of parameters and records the parameter synchronization task in the parameter synchronization task table.

[0069] In this example, when polling detects pending parameter change records, the global management node generates parameter synchronization tasks based on the pre-configured "number of synchronizations per session." For example, if the system is configured to synchronize a maximum of 10 records at a time, and there are currently 15 pending records, 10 tasks will be generated (the remaining 5 will be processed in the next poll). This controls the load on each synchronization session and prevents excessive system pressure. The generated tasks are recorded in the parameter synchronization task table. Each task includes information such as the specific content of the change and the target node, used for subsequent tracking of synchronization progress and verification of completion status. Through task-based management, discrete change operations are transformed into executable synchronization instructions, ensuring the orderliness and controllability of change processing.

[0070] The polling mechanism ensures timely response to parameter changes, the control of the number of synchronizations per cycle balances synchronization efficiency and system resource consumption, and the task record provides traceability for the synchronization process, ultimately improving the reliability and stability of parameter synchronization.

[0071] In one example, the synchronization task based on the parameters involves the management node distributing parameter change data to the target node, such as... Figure 3 As shown, it includes:

[0072] Step 301: The global management node sends the parameter change data to the microservice system configured in the parameter synchronization node information table.

[0073] In this example, the parameter synchronization node information table stores information about the microservice systems that need to participate in parameter synchronization, including key information such as the network address and type of each microservice system. The global management node queries this table to identify the target microservice systems that need to receive parameter change data, ensuring that parameter change data is only sent to the configured systems and avoiding invalid transmission.

[0074] Step 302: If the microservice system is deployed in a distributed manner, the parameter change data is sent from the global management node to the microservice management node, and the microservice management node further distributes the parameter change data to its subordinate distributed control nodes.

[0075] In this example, a distributed microservice system typically contains multiple subordinate nodes (such as a distributed control node, or DCN node for short), which need to be coordinated and synchronized through an intermediate node. The microservice management node is the only node in the distributed microservice system responsible for managing local parameters. The global management node first sends parameter change data to this management node, which then further distributes it to each subordinate DCN node. This hierarchical distribution mechanism adapts to the characteristics of distributed systems with multiple nodes and the need for coordination, avoiding network pressure caused by the global management node communicating directly with a large number of DCN nodes. At the same time, through the relay of the microservice management node, it ensures the consistency of parameter synchronization within the distributed system.

[0076] Step 303: If the microservice system is deployed in a centralized manner, the parameter change data is directly sent from the global management node to the microservice management node.

[0077] In this example, the centrally deployed microservice system has no independent management node; all its functions are handled directly by a single or a few instances. Therefore, the global management node can directly distribute parameter change data to the microservice management node itself without going through intermediate nodes. This direct distribution method simplifies the synchronization process of the centralized system, reduces latency in intermediate steps, and improves synchronization efficiency.

[0078] By accurately locating the target through the parameter synchronization node information table and combining differentiated distribution strategies for distributed and centralized deployments, the hierarchical coordination of the distributed system is guaranteed, while the synchronization efficiency of the centralized system is optimized, ultimately improving the parameter synchronization compatibility and reliability of the entire system in a hybrid deployment environment.

[0079] In one example, after the target node updates and synchronizes the parameter change data to the local parameter table, if the parameter is globally cached, such as... Figure 4 As shown, the method further includes:

[0080] Step 401: The target node generates a new parameter version number and updates the mapping relationship between the global version number and the local version number.

[0081] In this example, the globally cached parameters are critical parameters that require frequent access. To ensure consistency of parameter versions across different nodes, the target node needs to generate a new local parameter version number after updating its local parameter table. Simultaneously, since the global version number is a unified version identifier across multiple systems, the target node will associate the newly generated local version number with the current global version number and record it in a specific storage location (such as a parameter version mapping table). The purpose of this mapping is to allow for quick location of the local version on each node in subsequent operations using the global version number, ensuring cross-node version verification and synchronization tracking. By generating and mapping version numbers, a global and local version association is established, providing a basis for subsequent memory loading and version switching.

[0082] Step 402: After the target node updates the parameter change data to the local parameter table, all target nodes load the parameters into memory and switch versions according to the mapping relationship.

[0083] In this example, the globally cached parameters need to be loaded into memory to meet high-frequency access requirements. After all target nodes complete their local parameter table updates, they will identify the latest parameter version based on the version mapping relationship established in step 401 and load that version of the parameters into memory. After loading, the nodes need to switch the parameter version used in memory from the old version to the new version to ensure that subsequent business operations directly use the latest memory parameters. Memory loading improves parameter access efficiency, and version switching ensures that business operations use the latest parameters, avoiding business errors caused by database read latency or old version parameters.

[0084] Version number mapping ensures consistency between global and local versions, memory loading improves the access efficiency of high-frequency parameters, and version switching ensures that business operations use the latest parameters, ultimately achieving efficient synchronization and reliable use of global cache parameters.

[0085] In one example, all target nodes load parameters into memory, such as... Figure 5 As shown, it includes:

[0086] Step 501: The target node polls and detects the version upgrade status of the parameters.

[0087] In this example, the target node periodically checks the instance parameter version information table using a polling mechanism. The core purpose is to detect whether parameter versions have been upgraded in real time. Parameter version upgrades are typically triggered by a synchronization task initiated by the global management node. Once the local parameter table is updated, the version information is marked as "pending upgrade." Polling ensures that the node can detect version changes promptly, avoiding inconsistencies between memory parameters and database parameters due to lazy loading. By actively detecting version status, it provides triggering conditions for subsequent memory loading, ensuring the timeliness of parameter updates.

[0088] Step 502: The instance that wins the lock loads the latest parameters into memory and updates the status in the instance parameter version information table.

[0089] In this example, to avoid resource contention or data conflicts caused by multiple instances loading the same parameters simultaneously, the system employs a "locking mechanism": only the instance that successfully acquires the lock has the right to perform the actual memory loading operation. This instance reads the latest version of the parameters from its local parameter table and loads it into memory. After loading, the instance updates the status of its instance parameter version information table, recording the currently loaded version number for reference by other instances. This locking mechanism prevents data corruption caused by parallel loading from multiple instances, while also transmitting loading results through status updates.

[0090] Step 503: Instances that fail to acquire the lock obtain the latest version number based on the version upgrade status of the parameters and complete the memory loading.

[0091] In this example, instances that fail to acquire the lock do not need to repeat the loading operation. Instead, they directly obtain the latest version number by querying the instance parameter version information table to check the updated status. Based on this version number, these instances will extract the latest parameters from the multiple version copies already loaded in memory (or synchronize memory data through other efficient methods) to complete their own memory loading. This not only avoids resource waste but also ensures that all instances eventually use the latest parameters while maintaining business continuity.

[0092] In one example, the version switching based on the mapping relationship is as follows: Figure 6 As shown, it includes:

[0093] Step 601: After all target nodes have been successfully loaded, the parent management node of the target nodes issues a version switching instruction.

[0094] In this example, the prerequisite for version switching is that all target nodes have completed parameter loading: only when all target nodes have loaded the latest parameters into memory can the version be switched uniformly to avoid business logic confusion caused by some nodes using old version parameters. The upper-level management node (such as the microservice management node in a distributed microservice system or the global management node in a centralized microservice system) assumes the coordination responsibility at this stage, triggering all target nodes to synchronously execute the version switching operation by issuing version switching commands. For example, in a distributed system, the microservice management node sends a "switch to version V2" command to each of its subordinate DCN nodes; in a centralized system, the global management node directly sends the switching command to the microservice management node. This unified command ensures that all nodes use the latest parameters at the same time, guaranteeing the overall parameter consistency of the system.

[0095] Step 602: After the superior management node confirms that all target nodes have been successfully switched, update the parameter synchronization log table and the parameter synchronization task table status.

[0096] In this example, after issuing the switchover command, the upper-level management node needs to receive "switchover successful" status feedback from each target node. Only when all nodes report success is the version switchover confirmed as complete. At this point, the upper-level management node updates the parameter synchronization log table and the parameter synchronization task table. These status updates provide crucial information for subsequent process tracing and troubleshooting. For example, the log table can be used to see which node failed to switch over, or the task table can be used to confirm whether the synchronization was ultimately successful. Status recording and verification ensure closed-loop management of the version switchover process, avoiding potential risks caused by some nodes failing to switch over.

[0097] In one example, the parent management node of the target node issues a version switching instruction, including:

[0098] If the microservice system is deployed in a distributed manner, the microservice management node issues a version switching instruction to the distributed control node, and the distributed control node reports the completion status to the microservice management node after completing the switch.

[0099] In this example, for a distributed microservice system, the microservice management node acts as the coordination center within the system, responsible for the unified management of its subordinate distributed control nodes (DCN nodes). During version switching, the microservice management node issues a switchover command to each DCN node, ensuring that all subordinate nodes perform the switchover operation synchronously. After completing the switchover, each DCN node must report a "switchover complete" status to the microservice management node. This feedback mechanism ensures that the microservice management node can confirm whether each DCN node has successfully switched over, avoiding parameter inconsistencies caused by some nodes not switching over.

[0100] If the microservice system is deployed in a centralized manner, the microservice management node directly performs version switching and reports the completion status to the global management node.

[0101] In this example, for a centrally deployed microservice system, since there is no independent microservice management node, the microservice management node directly receives instructions from the global management node and completes the version switch independently. After the switch is completed, the microservice management node needs to directly report the "switch complete" status to the global management node, so that the global management node can directly grasp the switch result of the centralized system without going through an intermediate node.

[0102] In one example, the above data processing method is described in detail through a complete embodiment:

[0103] A. Parameters are maintained visually through the web management system (counter service). If the parameter maintenance transaction is sent to the global management node, then proceed to step B (if it is sent to the management node of the microservice system, then proceed to step B').

[0104] B. During the transaction process, if the global management node detects that the parameter table has been added, deleted or modified, it will generate the corresponding parameter change log and store it in the parameter change log table (tsp_param_change_log). If the parameter is a global cache, the memory parameters in the current version of the instance will also be updated.

[0105] C. The polling thread of the global management node checks the parameter change log (tsp_param_change_log) to determine whether a parameter synchronization task should be generated. If there is an onprocess record in the parameter change log table, a task will be generated based on the record configured in the application (how many records to synchronize at one time; if less than the configured number, one record will be generated; if more than the configured number, only the configured number will be taken, and the remaining records will be reserved for the next synchronization). (Global cache and non-global cache tasks are performed alternately. When the application starts, the non-global cache task is processed for the first time, and the synchronization record is also queried according to the task type). A record is inserted into the parameter synchronization task table (tsp_param_sync_task). Then, based on the node configured in the parameter synchronization node information table (tsp_param_sync_node), the corresponding number of synchronization logs are generated and inserted into the parameter synchronization log table (tsp_param_sync_log).

[0106] D. After generating the parameter synchronization task, the global management node calls the remote service according to the parameter synchronization node information table (tsp_param_sync_node) to send the records in the parameter change log table (tsp_param_change_log) to each microservice system node configured in the parameter synchronization node information table (tsp_param_sync_node), and the input parameters need to include the "global version number";

[0107] 1. If the microservice system is deployed in a distributed manner, the microservice management node needs to perform the parameter synchronization process for the microservices themselves:

[0108] B' The global management node sends the records in the parameter change log table (tsp_param_change_log) to the microservice management node and directly synchronizes the parameter synchronization log table (tsp_param_change_log). The microservice management node receives the "global version number". If the parameter is globally cached, it inserts the mapping relationship between the global version number and the microservice version number into the parameter version mapping table (tsp_param_version_relation).

[0109] C' The polling thread of the microservice management node checks the parameter change log table (tsp_param_change_log). If the parameter change log table has an onprocess record, it will generate a parameter synchronization task (similar to the global management node);

[0110] D' After generating the parameter synchronization task, the microservice management node calls the remote service according to the parameter synchronization node information table (tsp_param_sync_node) to send the records of the parameter change log table (tsp_param_change_log) to the DCN node configured in the parameter synchronization node information table (tsp_param_sync_node), and the input parameters need to include the "global version number";

[0111] E'. Each DCN node receives the "global version number", then updates and synchronizes the differences to the parameter table. If the parameter is globally cached, the parameter version number is upgraded and the mapping relationship between the global version number and the microservice version number is inserted into the parameter version mapping table (tsp_param_version_relation).

[0112] F' If the parameter is a global cache, each DCN node loads the parameter into memory through a polling thread; otherwise, it is not loaded.

[0113] G' If the parameter is globally cached, each DCN node (including ADM) needs to ensure that it has completed loading the parameter. Whether it is globally cached or not, it will report to the microservice management node and update the parameter version information table (tsp_instans_param_version) with the instance step "report_load_param_success - reported that loading the parameter was successful", and update the parameter synchronization log table (tsp_param_sync_log) with the corresponding DCN status.

[0114] 2. If the microservice system is deployed centrally, then there is no microservice management node:

[0115] E. The microservice management node receives the "global version number" (including the global management node) and directly updates and synchronizes the differences to the parameter table. If the parameter is a global cache, the parameter version number is upgraded and the mapping relationship between the global version number and the microservice version number is inserted into the parameter version mapping table (tsp_param_version_relation).

[0116] F. If the parameter is a global cache, each instance loads the parameter into memory through a polling thread;

[0117] G. For distributed microservice systems, the microservice management node needs to ensure that all DCN nodes have completed the parameter loading step; for centralized microservice systems, it needs to ensure that it itself has completed the parameter loading step. After the parameter loading step is completed, it reports to the global management node and updates the parameter synchronization log table (tsp_param_sync_log) to reflect the status of the corresponding microservice system.

[0118] H. After ensuring that all microservice systems have completed loading parameters, the global parameter management node needs to perform a second distribution if the parameters are globally cached, notifying each microservice management node to start switching versions; otherwise, parameter synchronization is complete.

[0119] 1. If the microservice system is deployed in a distributed manner, the microservice management node needs to perform the parameter synchronization process for the microservices themselves:

[0120] H', the microservice management node issues a second notification to each DCN node of the microservice to start switching versions;

[0121] I'. Each DCN node switches versions, ensuring that after completing the version switch, the instance step in the parameter version information table (tsp_instans_param_version) is "report_switch_version_success-report version switch as successful", and reports to the microservice management node, updating the corresponding DCN status in the parameter synchronization log table (tsp_param_sync_log);

[0122] 2. If the microservice system is deployed centrally, then there is no microservice management node:

[0123] The microservice management node performs version switching;

[0124] I. For distributed microservice systems, the microservice management node needs to ensure that all DCN nodes have completed the version switch; for centralized microservice systems, it needs to ensure that it itself has completed the version switch. After the version switch is complete, it reports to the global management node, updates the status of the corresponding microservice system in the parameter synchronization log table (tsp_param_sync_log), and updates the status of the parameter synchronization task table (tsp_param_sync_task). Global parameter synchronization is then complete.

[0125] This disclosure provides a data processing apparatus, such as Figure 7 As shown, the device includes:

[0126] The first generation module 701 is used for the global management node to receive parameter change requests, update global parameters according to the parameter change requests, generate parameter change logs and store them in the parameter change log table;

[0127] The second generation module 702 is used to generate a parameter synchronization task by the global management node based on the parameter change log, and record the parameter synchronization task in the parameter synchronization task table;

[0128] The distribution module 703 is used to synchronize tasks according to the parameters, and the global management node distributes the parameter change data to the target node; the target node includes a microservice management node or a distributed control node, and the parameter change data includes a parameter change log table and a global version number.

[0129] The update module 704 is used by the target node to update the parameter change data to the local parameter table.

[0130] In one example, the second generation module 702 is specifically used for:

[0131] The global management node polls the parameter change log table to determine whether there are any pending parameter change log records.

[0132] When there is a pending parameter change log record, the global management node generates a single synchronization task for the specified number of parameters and records the parameter synchronization task in the parameter synchronization task table.

[0133] In one example, distribution module 703 is specifically used for:

[0134] The global management node sends the parameter change data to the microservice system configured in the parameter synchronization node information table;

[0135] If the microservice system is deployed in a distributed manner, the parameter change data is sent from the global management node to the microservice management node, and the microservice management node further distributes the parameter change data to its subordinate distributed control nodes;

[0136] If the microservice system is deployed in a centralized manner, the parameter change data is directly sent from the global management node to the microservice management node.

[0137] In one example, the device further includes: a third generation module 705 and a switching module 706; after the target node updates and synchronizes the parameter change data to the local parameter table, when the parameter is globally cached:

[0138] The third generation module 705 is used to generate a new parameter version number for the target node, and at the same time update the mapping relationship between the global version number and the local version number.

[0139] The switching module 706 is used to load parameters into memory and perform version switching according to the mapping relationship after the target node updates the parameter change data to the local parameter table.

[0140] In one example, the switching module 706 is specifically used when all target nodes load parameters into memory to:

[0141] The target node polls to detect the version upgrade status of its parameters;

[0142] The instance that wins the lock loads the latest parameters into memory and updates the status in the instance parameter version information table;

[0143] Instances that fail to acquire the lock obtain the latest version number based on the version upgrade status parameters and complete the memory loading.

[0144] In one example, the switching module 706 is specifically used for: switching versions according to the mapping relationship.

[0145] After all target nodes have been successfully loaded, the parent management node of the target nodes issues a version switching instruction;

[0146] After the superior management node confirms that all target nodes have been successfully switched, it updates the status of the parameter synchronization log table and the parameter synchronization task table.

[0147] In one example, the switching module 706 is specifically used when the target node's parent management node issues a version switching command:

[0148] If the microservice system is deployed in a distributed manner, the microservice management node issues a version switching instruction to the distributed control node, and the distributed control node reports the completion status to the microservice management node after completing the switch.

[0149] If the microservice system is deployed in a centralized manner, the microservice management node directly performs version switching and reports the completion status to the global management node.

[0150] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0151] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0152] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0153] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0154] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0155] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0156] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0157] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0158] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0159] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0160] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0161] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0162] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0163] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: The global management node receives parameter change requests, updates global parameters according to the parameter change requests, generates parameter change logs, and stores them in the parameter change log table; Based on the parameter change log, the global management node generates a parameter synchronization task and records the parameter synchronization task in the parameter synchronization task table; According to the parameter synchronization task, the global management node distributes the parameter change data to the target node; the target node includes a microservice management node or a distributed control node, and the parameter change data includes a parameter change log table and a global version number. The target node updates the parameter change data to its local parameter table; Based on the parameter change log, the global management node generates a parameter synchronization task and records the parameter synchronization task in the parameter synchronization task table, including: The global management node polls the parameter change log table to determine whether there are any pending parameter change log records. When there is a pending parameter change log record, the global management node generates a single synchronization task for the specified number of parameters and records the parameter synchronization task in the parameter synchronization task table. The step of synchronizing tasks based on the parameters, in which the management node distributes parameter change data to the target node, includes: The global management node sends the parameter change data to the microservice system configured in the parameter synchronization node information table; If the microservice system is deployed in a distributed manner, the parameter change data is sent from the global management node to the microservice management node, and the microservice management node further distributes the parameter change data to its subordinate distributed control nodes; If the microservice system is deployed in a centralized manner, the parameter change data is directly sent from the global management node to the microservice management node.

2. The method according to claim 1, characterized in that, After the target node updates and synchronizes the parameter change data to the local parameter table, when the parameter is globally cached, the method further includes: The target node generates a new parameter version number and updates the mapping relationship between the global version number and the local version number. After the target node updates the parameter change data to its local parameter table, all target nodes load the parameters into memory and switch versions according to the mapping relationship.

3. The method according to claim 2, characterized in that, All target nodes load parameters into memory, including: The target node polls to detect the version upgrade status of its parameters; The instance that wins the lock loads the latest parameters into memory and updates the status in the instance parameter version information table; Instances that fail to acquire the lock obtain the latest version number based on the version upgrade status parameters and complete the memory loading.

4. The method according to claim 2, characterized in that, The version switching based on the mapping relationship includes: After all target nodes have been successfully loaded, the parent management node of the target nodes issues a version switching instruction; After the superior management node confirms that all target nodes have been successfully switched, it updates the status of the parameter synchronization log table and the parameter synchronization task table.

5. The method according to claim 4, characterized in that, The target node's parent management node issues a version switching command, including: If the microservice system is deployed in a distributed manner, the microservice management node issues a version switching instruction to the distributed control node, and the distributed control node reports the completion status to the microservice management node after completing the switch. If the microservice system is deployed in a centralized manner, the microservice management node directly performs version switching and reports the completion status to the global management node.

6. A data processing apparatus, characterized in that, The device includes: The first generation module is used for the global management node to receive parameter change requests, update global parameters according to the parameter change requests, generate parameter change logs and store them in the parameter change log table; The second generation module is used to generate parameter synchronization tasks by the global management node based on the parameter change log, and record the parameter synchronization tasks in the parameter synchronization task table; The distribution module is used to synchronize tasks according to the parameters. The global management node distributes the parameter change data to the target node. The target node includes a microservice management node or a distributed control node. The parameter change data includes a parameter change log table and a global version number. The update module is used by the target node to update the parameter change data to the local parameter table; The second generation module is specifically used for: the global management node polling the parameter change log table to determine whether there are any pending parameter change log records; when there are pending parameter change log records, the global management node generates a single synchronization number of parameter synchronization tasks and records the parameter synchronization tasks in the parameter synchronization task table; The distribution module is specifically used for: the global management node sending the parameter change data to the microservice system configured in the parameter synchronization node information table; if the microservice system is deployed in a distributed manner, the parameter change data is sent from the global management node to the microservice management node, and the microservice management node further distributes the parameter change data to its subordinate distributed control nodes; if the microservice system is deployed in a centralized manner, the parameter change data is sent directly from the global management node to the microservice management node.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

Images