Configuration data issuing method, device and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a configuration data issuing method, device and system, electronic equipment and a storage medium, the method is applied to a configuration transfer node in a configuration distribution system, and the configuration distribution system further comprises a configuration management end, message middleware and a plurality of service nodes used for providing target services. The method comprises the following steps: acquiring a configuration updating message issued by a configuration management end from message middleware; wherein the configuration update message comprises update data of at least one configuration data block, and the configuration data block is obtained by splitting overall configuration data of the target service by the configuration management end; storing update data of a configuration data block included in the configuration update message into a persistent storage device and a cache device; and in response to a configuration acquisition request initiated by a target service node in the plurality of service nodes, acquiring update data of the configuration data block from the cache device or the persistent storage device, and feeding back the update data of the configuration data block to the target service node.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, apparatus, system, electronic device and storage medium for distributing configuration data. Background Technology

[0002] In microservice and distributed systems, it is often necessary to distribute unified service configurations to a massive number of service nodes. In implementing this invention, the inventors discovered that traditional service configuration distribution schemes have the following problems: (1) The full distribution method is usually adopted. Even if there are only a few configuration changes, the complete configuration data needs to be redistributed, which leads to a serious waste of network bandwidth and node processing resources.

[0003] (2) Service nodes typically pull configuration data from the central server directly or through a simple proxy. When a large number of nodes make requests simultaneously, the central server and the backend database are prone to significant concurrency pressure, leading to response delays. Summary of the Invention

[0004] This invention provides a configuration data distribution method, apparatus, system, electronic device, and storage medium, which can reduce resource consumption, improve the efficiency of configuration data distribution, and ensure system stability and high performance.

[0005] In a first aspect, the configuration data distribution method provided in this embodiment of the invention is applied to a configuration relay node in a configuration distribution system. The configuration distribution system further includes a configuration management terminal, a message middleware, and multiple service nodes for providing target services. The method includes: Retrieve configuration update messages published by the configuration management end from the message middleware; wherein, the configuration update message includes update data of at least one configuration data block, and the configuration data block is obtained by the configuration management end after splitting the overall configuration data of the target service; The updated data of the configuration data blocks included in the configuration update message is stored in a persistent storage device and a cache device; In response to a configuration retrieval request initiated by the target service node among multiple service nodes, the updated configuration data block is retrieved from the cache device or persistent storage device, and the updated configuration data block is fed back to the target service node.

[0006] Secondly, the configuration data distribution device provided in this embodiment of the invention is set in a configuration relay node of a configuration distribution system. The configuration distribution system also includes a configuration management terminal, a message middleware, and multiple service nodes for providing target services. The device includes: The acquisition module is used to obtain configuration update messages published by the configuration management terminal from the message middleware. The configuration update message includes update data of at least one configuration data block, which is obtained by the configuration management terminal after splitting the overall configuration data of the target service. The storage module is used to store the updated data of the configuration data blocks included in the configuration update message into a persistent storage device and a cache device; The distribution module is used to respond to configuration retrieval requests initiated by the target service node among multiple service nodes, retrieve the updated configuration data block from the cache device or persistent storage device, and feed back the updated configuration data block to the target service node.

[0007] Thirdly, the configuration distribution system provided in the embodiments of the present invention includes a configuration relay node for executing the configuration data distribution method of any embodiment of the present invention, as well as a configuration management terminal, a message middleware, and multiple service nodes for providing target services.

[0008] Fourthly, the electronic device provided in the embodiments of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the configuration data distribution method as described in any embodiment of the present invention.

[0009] Fifthly, the computer-readable storage medium provided in the embodiments of the present invention stores a computer program thereon, which, when executed by a processor, implements the configuration data distribution method as described in any embodiment of the present invention.

[0010] In this embodiment of the invention, a configuration data chunking mechanism is introduced, and only the changed module data is encapsulated in the configuration update message. This achieves precise incremental update delivery, reduces the amount of data transmitted over the network, and lowers the processing overhead of the configuration relay node and the parsing overhead of the service node, making high-frequency, localized configuration updates more efficient. A message middleware is introduced as the channel for configuration update publishing and subscription, and a configuration relay node is established as an intermediate layer. The message middleware decouples the publisher (configuration management end) and the consumer (configuration relay node), and has the ability to smooth out traffic peaks and valleys, thus smoothing out sudden large-scale configuration update traffic and avoiding direct impact on downstream systems. The introduction of the configuration relay node distributes the pressure of the central server to multiple horizontally scalable intermediate nodes, eliminating single-point bottlenecks and alleviating the concurrent pressure faced by the central server and the backend database. Data is stored simultaneously in a persistent storage device and a cache device. The cache device provides high-speed response for query requests from massive service nodes, greatly improving read performance. The persistent storage device ensures reliable data persistence. When the cache misses, the request can be returned to the persistent storage. The two-level storage ensures system stability and high performance. Attached Figure Description

[0011] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a configuration data distribution method provided in an embodiment of the present invention; Figure 2 This is another flowchart illustrating the configuration data distribution method provided in this embodiment of the invention; Figure 3 This is an example diagram of the configuration data distribution method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a configuration data distribution device provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] Figure 1This is a flowchart illustrating a configuration data distribution method provided in an embodiment of the present invention. This method is applicable to scenarios involving the distribution of configuration data to a massive number of service nodes. The configuration data distribution method can be executed by a configuration data distribution device provided in this embodiment, which can be implemented in software and / or hardware. In a specific embodiment, this device can be integrated into an electronic device, which acts as a configuration relay node and can be a server, a cloud computing platform instance, etc. The following embodiment uses the integration of the configuration data distribution device into an electronic device as an example for illustration. (See also...) Figure 1 The configuration data distribution method in this embodiment may include the following steps: Step 101: Obtain the configuration update message published by the configuration management terminal from the message middleware.

[0016] The configuration update message includes update data for at least one configuration data block, which is obtained by splitting the overall configuration data of the target service by the configuration management end.

[0017] A service can be understood as an independent software functional unit that provides specific business functions. A target service refers to the specific application service type to which the overall configuration data belongs. Target services are typically used to implement specific business logic. For example, a target service could be a routing and forwarding service, an order service, etc. A routing and forwarding service is responsible for determining how network traffic is distributed among different backend instances, user access control, Internet Protocol (IP) address allocation, etc. Taking a routing and forwarding service as an example, as user traffic changes, the service will scale up or down, requiring configuration updates. Therefore, the service needs to obtain the latest configuration to process business normally. The configuration management terminal can be understood as the computing node responsible for generating, splitting, publishing, and managing the updated overall configuration data of the target service. Overall configuration data refers to the complete set of configuration information for a specific target service, covering all configuration items required for the target service to run. A configuration item can be understood as the most basic, indivisible logical unit in the configuration data, used to define or control a specific behavior, parameter, or rule of the target service. Examples include timeout thresholds for controlling payment processes, Boolean switches to enable specific functions, or endpoint addresses for connecting to external services.

[0018] A configuration relay node can be understood as a computing unit used to receive, store, and distribute configuration data between the target service node and the configuration management terminal. Specifically, a configuration relay service is configured on the configuration relay node, which can be used to execute the configuration data distribution method in this embodiment of the invention.

[0019] Configuration update messages can be understood as messages published by the configuration management terminal to various configuration relay nodes to indicate configuration updates. Specifically, configuration update messages may contain identifiers of configuration data blocks, updated data for the configuration data blocks, and version numbers of the configuration data blocks.

[0020] A configuration data block is a subset of configuration information that can be independently updated, obtained by splitting the overall configuration data according to a specific dimension. Each configuration data block contains at least one configuration item. Typically, each configuration data block has independent business semantics. Specifically, the configuration management end can logically group configuration items according to their function in the business scenario. For example, if the target service is an order service, the configuration data blocks may include payment business configuration blocks, logistics business configuration blocks, etc. The payment business configuration block contains configuration items such as payment timeout time and payment method parameters; the logistics business configuration block contains configuration items such as freight calculation rule parameters and delivery channels. After splitting the overall configuration data of the target service, the updated configuration data blocks can be determined based on the scope of business affected by the update. When publishing a configuration update message, only the updated configuration data blocks are published, without publishing the overall configuration data of the target service. Each configuration data block can have a corresponding version number to identify the current version of the configuration data block and to trace its historical version changes. The overall configuration data and configuration data blocks are typically stored and transmitted in a structured data format.

[0021] The updated configuration data block refers to the set of configuration data that needs to be distributed and used to update the target service nodes in this configuration distribution process; it serves as the carrier for configuration updates during the release process. Specifically, the configuration management end splits the overall configuration data of the target service into multiple independent configuration data blocks. When a configuration update is required, it updates some or all of the configuration items in the overall configuration data of the target service, or adds or deletes some configuration items, depending on the needs. Each update generates a unique version identifier for each configuration data block containing the updated configuration items. Subsequently, a configuration update message is generated, which includes the updated configuration data block and the version identifier bound to it. Finally, the configuration update message is published to the configuration distribution nodes through a message middleware.

[0022] In this embodiment, a configuration data segmentation mechanism is introduced, and only the changed module data is encapsulated in the configuration update message. This enables precise incremental update delivery, reduces the amount of data transmitted over the network, lowers the processing overhead of configuration relay nodes and the parsing overhead of service nodes, and makes high-frequency, local configuration updates more efficient.

[0023] Message middleware can be understood as a computing unit used for asynchronous transmission of configuration update messages between configuration relay nodes and configuration management terminals. Specifically, message middleware is typically deployed on a server cluster consisting of multiple servers, employing a publish-subscribe model. The configuration management terminal acts as the publisher, publishing configuration update messages to the message middleware, while the configuration relay nodes act as subscribers, retrieving configuration update messages from subscribed message topics from the message middleware, thus decoupling the publisher (configuration management terminal) and the consumer (configuration relay nodes). Each message topic corresponds to a message queue. Once the publisher publishes a configuration update message to the message topic, it can return immediately without waiting for consumer processing, achieving asynchronous communication. When a sudden surge in configuration update messages occurs, the message queue acts as a buffer to temporarily store messages. Consumers can pull configuration update messages at a uniform rate according to their processing capacity, avoiding instantaneous traffic spikes and thus achieving traffic smoothing. Multiple configuration relay nodes can form consumer groups to achieve load balancing and parallel processing. The system maintains consumption offsets to accurately record read positions, ensuring that messages are not missed or duplicated, thereby achieving reliable and orderly asynchronous communication.

[0024] In this embodiment, a message middleware is introduced as a channel for configuration update publishing and subscription, and a configuration relay node is established as an intermediate layer. The message middleware decouples the publisher (configuration management end) and the consumer (configuration relay node), and has the ability to smooth out traffic peaks and valleys, which can smooth out sudden large-scale configuration update traffic and avoid directly impacting downstream systems. The introduction of the configuration relay node distributes the pressure of the central server to multiple horizontally scalable intermediate nodes, eliminates single point bottlenecks, and alleviates the concurrent pressure faced by the central server and the backend database.

[0025] Step 102: Store the updated data of the configuration data block included in the configuration update message into a persistent storage device and a cache device.

[0026] Persistent storage devices can be understood as storage units capable of storing data for a long time and ensuring that the data is not lost after a restart. Examples of persistent storage devices include relational database servers and distributed file systems. Caching devices can be understood as high-speed memory-based data storage units, where stored data may be lost after a restart. Specifically, caching devices have fast read / write speeds but are limited by small memory capacity and are typically volatile. In contrast, persistent storage devices are disk-based storage systems, characterized by strong data persistence, large storage capacity, and relatively low cost, but slower read / write speeds. Therefore, persistent storage devices and caching devices need to work together, storing updated configuration data blocks to ensure data security.

[0027] Specifically, the updated data of the configuration data block can be stored in both persistent storage and cache. When querying the updated data of the configuration data block, the cache is accessed to achieve fast reading. If the updated data of the required configuration data block cannot be found in the cache, it means that the updated data of some configuration data blocks stored in the cache has expired or been deleted. In this case, the persistent storage can still be queried to ensure that the updated data of the required configuration data block can be successfully read.

[0028] In this embodiment, by storing data simultaneously in a persistent storage device and a cache device, the cache device provides a high-speed response to query requests from a large number of service nodes, greatly improving read performance; the persistent storage device ensures reliable data persistence; by obtaining updated configuration data blocks from the cache device or the persistent storage device, when the cache misses, the request can be sent back to the persistent storage. The two-level storage ensures system stability and high performance.

[0029] Step 103: Respond to the configuration retrieval request initiated by the target service node among multiple service nodes, retrieve the updated data of the configuration data block from the cache device or persistent storage device, and feed back the updated data of the configuration data block to the target service node.

[0030] A target service node can be understood as a specific computing unit that has deployed a target service instance and initiated a configuration retrieval request. Specifically, a target service node can directly provide specific business processing capabilities to users or other services.

[0031] Specifically, the target service can periodically send requests to the configuration management terminal or message middleware to obtain the latest version number of each configuration data block in the overall configuration data, and compare it with the current version number of its own configuration data block. If they are inconsistent, the target service initiates a configuration retrieval request to the configuration distribution node corresponding to the target service to obtain the updated configuration data block data to update its own current configuration data block. The configuration retrieval request may include the identifier of the target service node, the identifier of the configuration data block, and the version number, etc. In response to the configuration retrieval request from the target service node, the configuration distribution node will query the updated configuration data block data in the cache device or persistent storage device. In this embodiment, there are no restrictions on whether to query the cache device or the persistent storage device, or the order of the queries. For example, the system can first query whether the updated configuration data block exists in the cache device to quickly obtain the required updated configuration data block. If not found, it can then query whether the updated configuration data block exists in the persistent storage device and feed back the updated configuration data block to the target service node. In addition, in order to distribute access traffic, some configuration relay nodes responsible for distributing configurations for hot application services can prioritize querying the cache device, while other configuration relay nodes can prioritize querying the persistent storage device.

[0032] In this embodiment, a configuration data chunking mechanism is introduced, and only the changed module data is encapsulated in the configuration update message. This enables precise incremental update delivery, reduces the amount of data transmitted over the network, and lowers the processing overhead of the configuration relay node and the parsing overhead of the service node, making high-frequency, localized configuration updates more efficient. A message middleware is introduced as the channel for configuration update publishing and subscription, and a configuration relay node is established as an intermediate layer. The message middleware decouples the publisher (configuration management end) and the consumer (configuration relay node), and has the ability to smooth out traffic peaks and valleys, thus smoothing out sudden large-scale configuration update traffic and avoiding direct impact on downstream systems. The introduction of the configuration relay node distributes the pressure of the central server to multiple horizontally scalable intermediate nodes, eliminating single-point bottlenecks and alleviating the concurrent pressure faced by the central server and the backend database. Data is stored simultaneously in a persistent storage device and a cache device. The cache device provides high-speed response for query requests from massive service nodes, greatly improving read performance. The persistent storage device ensures reliable data persistence. When the cache misses, the request can be returned to the persistent storage. The two-level storage ensures system stability and high performance.

[0033] The following further explains the configuration data distribution method provided in the embodiments of the present invention, such as... Figure 2 As shown, Figure 2 This is another flowchart illustrating the configuration data distribution method provided in this embodiment of the invention. The configuration data distribution method in this embodiment may include: Step 201: Subscribe to the message topics of the message middleware according to the business dimensions that the relay node is responsible for.

[0034] The business dimensions handled by a configuration relay node can be understood as the basis for grouping and managing service nodes according to business-related standards in the configuration distribution system. Typically, a configuration relay node is responsible for service nodes within one or more specific business dimensions. A message topic can be understood as an entity in the message middleware used to classify and organize messages. Message topics usually adopt a hierarchical organizational structure, such as "domain: subdomain". Each message topic corresponds to a specific message stream. Publishing refers to the configuration management end sending configuration update messages to its corresponding message topic; subscribing refers to the configuration relay node receiving a set of messages of interest by subscribing to topics. This enables decoupled communication based on update message content, preventing irrelevant configuration update messages from reaching the configuration relay node.

[0035] Specifically, the business dimensions handled by a configuration node are related to the message topics it subscribes to. For example, message topics can be categorized by user group and / or region. If configuration relay node 1 is a service node responsible for payment services for members in City A, then the message topic subscribed to by configuration relay node 1 would be "Member User Configuration Update: City A Payment Service Configuration Update".

[0036] In this embodiment, by subscribing to message topics of the message middleware according to the business dimension of the configuration relay node, it is possible to filter and accurately guide the traffic of the message middleware, reduce the communication overhead of the configuration distribution system, and avoid erroneous configuration changes from affecting other services, thereby enhancing the system's fault tolerance and recovery capabilities.

[0037] Step 202: Listen to the message topic and obtain the configuration update message published to the message topic.

[0038] Specifically, configuration relay nodes establish listening relationships for one or more message topics by subscribing to message topics in the message middleware. The message middleware can adopt a publish-subscribe consumption mechanism. When the configuration management end (producer) sends a configuration update message for a certain message topic to the message middleware, if that message topic is a topic subscribed to by the configuration relay node, the configuration relay node will listen for this update message and pull the newly arrived configuration update message from the message middleware.

[0039] Step 203: Store the updated data of the configuration data block included in the configuration update message into a persistent storage device and a cache device.

[0040] Step 204: In response to the configuration retrieval request initiated by the target service node, first query the cache device to obtain the updated data of the configuration data block.

[0041] Specifically, since the query speed of the cache device is higher than that of the persistent storage device, the cache device can be queried first to quickly obtain the updated data of the configuration data block.

[0042] Optionally, the configuration retrieval request is initiated by the target service node according to a discrete update cycle, which causes the configuration retrieval requests of multiple service nodes to be distributed in time.

[0043] The update period can be understood as the time interval between two consecutive configuration retrieval requests initiated by the target service node. A discrete update period can be understood as an update period of varying length, allowing different service nodes to initiate configuration requests at different time intervals. Specifically, a fixed initial update period and a random time offset window can be set. The specific time point at which the target service node initiates each configuration retrieval request is the initial update period plus the random time offset. For example, if the fixed initial update period is 30 seconds and the random time offset can be between 0 and 10 seconds, after the target service node completes a configuration update, if the random time offset is set to 7 seconds, it will initiate another configuration retrieval request to the configuration relay node after 37 seconds.

[0044] In this embodiment, the configuration retrieval request is initiated according to a discrete update cycle, which can keep the access traffic of each node stable and controllable, avoid the resonance effect caused by a large number of service nodes simultaneously initiating configuration retrieval requests, and prevent the configuration relay node and persistent storage device from crashing due to traffic overload, thereby improving the security of the configuration distribution system.

[0045] The configuration retrieval request is initiated by the target service node after comparing the version identifier of the configuration data block and determining that an incremental update of the configuration data block is required.

[0046] Specifically, when the target service node sends a configuration retrieval request to the configuration relay node, the configuration retrieval request may include the identifier of the target service node, the identifier of the configuration data block and the version number, as well as the update mode of the configuration data block. The update mode of the configuration data block may include incremental update and full update.

[0047] Correspondingly, when the target service node determines that an incremental update of the configuration data block is required, the updated data of the configuration data block is the incremental update data, which is used to indicate the data changes of the latest version of the configuration data block compared to the previous version.

[0048] Incremental update data can be understood as a data set containing only the changed configuration item data in the configuration data block. When updating the configuration based on incremental update data, the changed configuration items can be added, deleted, or modified using the configuration item identifier to achieve the configuration update. For example, if the "Order Timeout Configuration Block" contains four configuration items: Payment Timeout, Shipping Timeout, Automatic Cancellation Timeout, and Customer Service Intervention Timeout, and if the current requirement is to adjust the payment timeout from 30 seconds to 45 seconds, the incremental data can only contain the name of the payment timeout configuration item and the updated data "45 seconds".

[0049] Optionally, when the update mode is full update, the updated data in the configuration data block is full update data. Full update data can be understood as a data set containing all configuration item data in the configuration data block. That is, full update data includes both data for configuration items that have changed in this configuration update and data for configuration items that have not changed. When performing a configuration update based on full data, the current configuration data of the target service can be directly replaced with the full data to achieve the configuration update. Full update data is suitable for situations where the configuration data block is small, the target service is being configured for the first time, or there are many changed configuration items. Continuing the previous example, even if only the payment timeout configuration item is updated, the full update data will still contain data for all four configuration items in the "order timeout configuration block".

[0050] In this embodiment, incremental updates to the configuration data blocks can avoid consuming computing resources when assembling configuration files, reduce data transmission volume, and shorten the data latency of configuration distribution.

[0051] Step 205: Determine if the cache device has updated data for the configuration data block. If it does, proceed to step 209; otherwise, proceed to step 206.

[0052] Specifically, if the updated configuration data block is stored in the cache device, the updated configuration data block can be directly obtained and fed back to the target service node, thus achieving rapid configuration updates.

[0053] Because the storage capacity of the cache device is limited and the updated data of the configuration data blocks stored therein is volatile, there may be situations where the updated data of the configuration data blocks cannot be found in the cache device. In this case, it is necessary to query it again in the persistent storage device.

[0054] Step 206: Determine whether the query volume of the persistent storage device has reached the preset limit. If it has, proceed to step 210; otherwise, proceed to step 207.

[0055] The query volume of persistent storage devices can be understood as a metric that quantifies the access pressure exerted on persistent storage devices by query requests aimed at retrieving specified configuration data. Query volume can be measured in terms of quantity or traffic. For example, the query volume of persistent storage devices can be the cumulative number of query requests within one minute (quantity) or the number of query requests per second (traffic).

[0056] A preset limit can be understood as a maximum value that is pre-set for the access volume of persistent storage devices, allowing subsequent accesses. When the query volume of persistent storage devices reaches the preset limit, it indicates that the access pressure on persistent storage devices is too high and there is a risk of overload. Therefore, persistent storage devices will reject or temporarily restrict subsequent query requests from configuration relay nodes. Accordingly, when the query volume is a quantity, the preset limit can be a quantity limit; when the query volume is a flow, the preset limit can be a flow limit. For example, if the query volume is identified by the cumulative number of query requests within one minute, the quantity limit can be set to 10,000 times.

[0057] Step 207: Query the persistent storage device to obtain the update data of the configuration data block.

[0058] Specifically, when the number of queries to the persistent storage device does not reach the preset limit, the query requests will not be restricted, so the updated data of the configuration data block can be obtained directly by querying the persistent storage device.

[0059] Step 208: Write the updated data of the queried configuration data block into the cache device.

[0060] Specifically, after retrieving the updated configuration data block from the persistent storage device, step 209 can be executed first to send the updated configuration data block back to the target service node to ensure timely configuration distribution. Then, the retrieved updated configuration data block is written to the cache device. Writing the updated configuration data block to the cache device allows other service nodes to directly retrieve the updated configuration data block from the cache device when needed, shortening the data retrieval time.

[0061] Step 209: Feed back the updated configuration data block to the target service node.

[0062] Specifically, the target service node can update its current configuration data based on the updated data of the obtained configuration data block. The specific update method varies depending on whether the updated data of the configuration data block is an incremental update or a full update. After the update is completed, the new configuration will take effect, and the target service node will change the current version number of its own configuration data block to the version number corresponding to the updated configuration data block.

[0063] Step 210: Deny query for persistent storage.

[0064] Optionally, after rejecting the query for persistent storage, the configured relay node can wait for a certain period of time before initiating another query for persistent storage.

[0065] In this embodiment, a preset limit is set for the query volume of the persistent storage device, which can prevent the persistent storage device from becoming a performance bottleneck or causing service unavailability due to instantaneous high concurrency reads, thus maintaining the security and stability of the configuration distribution system.

[0066] In this embodiment, a configuration data block mechanism is introduced, and only the changed module data is encapsulated in the configuration update message. This achieves precise incremental update delivery, reduces the amount of data transmitted over the network, lowers the processing overhead of the configuration relay node, and reduces the parsing overhead of the service node, making high-frequency, localized configuration updates more efficient. Subscribing to message topics in the message middleware based on the business dimension handled by the configuration relay node enables filtering and precise guidance of message middleware traffic, reducing the communication overhead of the configuration distribution system. It also prevents erroneous configuration changes from affecting other services, enhancing the system's fault tolerance and recovery capabilities. Configuration retrieval requests are initiated according to discrete update cycles, ensuring stable and controllable access traffic for each node. This avoids a large number of target service nodes simultaneously initiating configuration retrieval requests, preventing the configuration relay node and persistent storage device from crashing due to traffic overload, thus improving the security of the configuration distribution system. Incremental updates of configuration data blocks can avoid... The configuration file assembly-free approach consumes less computing resources, reduces data transmission volume, and shortens the data latency of configuration distribution. Data is simultaneously stored in persistent storage and a cache. The cache provides high-speed response to query requests from a large number of service nodes, greatly improving read performance. The persistent storage ensures reliable data persistence. Updated configuration data blocks are retrieved from either the cache or persistent storage. If the cache misses, the request can be routed back to persistent storage; this two-tiered storage ensures system stability and high performance. Writing the updated configuration data blocks to the cache allows other nodes to directly retrieve the updated data from the cache, shortening the data retrieval time. Setting preset limits on the query volume of persistent storage prevents it from becoming a performance bottleneck or causing service unavailability due to sudden high concurrency reads, thus maintaining the security and stability of the configuration distribution system.

[0067] Figure 3 This is an example diagram of the configuration data distribution method provided in an embodiment of the present invention, such as... Figure 3 As shown, the configuration distribution system includes a configuration relay node, a configuration management terminal, a message middleware, and multiple service nodes for providing target services. These service nodes can all initiate configuration update requests based on discrete update cycles.

[0068] The configuration management system updates the configuration data of the target service based on business needs. Specifically, the configuration management system splits the overall configuration data of the target service into multiple independent configuration data blocks according to specific dimensions. When the configuration management system needs to publish a configuration update, it sends a configuration update message to each configuration relay node. The configuration update message contains the updated data of the changed configuration data blocks and their corresponding version numbers. However, the configuration update message does not directly reach each configuration relay node. Instead, it is first published to the corresponding message topic in the message middleware. The configuration relay nodes subscribe to and listen to specific message topics in the message middleware to obtain configuration update messages from the subscribed message topics. After a configuration relay node pulls a configuration update message from a subscribed message topic, the message middleware modifies the consumption offset to achieve ordered asynchronous communication. After obtaining the configuration update message, the configuration relay node parses the updated data of the configuration data blocks in the configuration update message and stores it simultaneously in persistent storage and cache, waiting to query it when the target service node receives a configuration retrieval request.

[0069] When the discrete update cycle arrives, the target service node retrieves the latest version number of all configuration data blocks in the overall configuration data of the target service from the configuration relay node or the configuration management terminal, and compares it with the current version number of its own configuration data blocks. If at least one configuration data block's version number is inconsistent, it indicates that the target service's configuration has been updated. In this case, it needs to send a configuration retrieval request to the configuration relay node to update the corresponding configuration data block to the latest version. Afterwards, the target service node can decide the update mode required for this configuration update. If the update mode is incremental, it needs to retrieve the incremental data of the configuration data blocks; if the update mode is full update, it needs to retrieve the full update data of the configuration database. After determining the update mode, the target service node sends a configuration retrieval request to the configuration relay node. The configuration retrieval request may include the target service node's identifier, the identifier and version number of the required configuration data blocks, the update mode, and other information. After receiving a configuration retrieval request from the target service node it is responsible for, the configuration relay node first parses the identifier, version number, and update mode of the configuration data block contained in the configuration retrieval request. It then attempts to query the cache device for the updated configuration data block. If the cache device contains the updated configuration data block, it can directly send the retrieved updated configuration data block back to the target service node, enabling the target service node to update its own configuration. If the cache device does not contain the updated configuration data block, it attempts to query the persistent storage device. If the query request is not rate-limited, the updated configuration data block can be found in the persistent storage device and sent back to the target service node. However, if the query request is rate-limited, it can query again after a period of time and return a "query not executed" message.

[0070] Figure 4 This is a schematic diagram of a configuration data distribution device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the device includes: The acquisition module 401 is used to acquire configuration update messages published by the configuration management terminal from the message middleware; wherein, the configuration update message includes update data of at least one configuration data block, and the configuration data block is obtained by the configuration management terminal after splitting the overall configuration data of the target service; Storage module 402 is used to store the update data of the configuration data block included in the configuration update message into a persistent storage device and a cache device; The distribution module 403 is used to respond to the configuration retrieval request initiated by the target service node among multiple service nodes, retrieve the updated data of the configuration data block from the cache device or persistent storage device, and feed back the updated data of the configuration data block to the target service node.

[0071] In one embodiment, the acquisition module 401 acquires the configuration update message published by the configuration management terminal from the message middleware, including: Subscribe to message topics in the message middleware based on the business dimensions that the relay node is responsible for; Listen to the message topic and retrieve the configuration update messages published to the message topic.

[0072] In one embodiment, the distribution module 403 responds to a configuration retrieval request initiated by the target service node and retrieves updated configuration data blocks from a cache device or persistent storage device, including: In response to the configuration retrieval request initiated by the target service node, first query the cache device to obtain the updated data of the configuration data block; If the cache device does not have updated data for the configuration data block, query the persistent storage device to obtain the updated data for the configuration data block.

[0073] In one embodiment, after the distribution module 403 obtains the update data of the configuration data block from the persistent storage device, it further includes: The updated data of the queried configuration data block is written to the cache device.

[0074] In one embodiment, the issuing module 403 queries the persistent storage device to obtain the update data of the configuration data block, including: Determine whether the query volume on the persistent storage device has reached a preset limit; If the number of queries to the persistent storage device does not reach the preset limit, query the persistent storage device to obtain the updated data of the configuration data block.

[0075] In one embodiment, the updated data of the configuration data block is incremental update data, which is used to indicate the data changes of the latest version of the configuration data block compared to the previous version.

[0076] In one embodiment, the configuration retrieval request is initiated by the target service node after comparing the version identifier of the configuration data block and determining that an incremental update of the configuration data block is required.

[0077] In one embodiment, the configuration retrieval request is initiated by the target service node according to a discrete update cycle, which causes the configuration retrieval requests of the multiple service nodes to be distributed in time.

[0078] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0079] The device in this embodiment introduces a configuration data block mechanism and encapsulates only the changed module data in the configuration update message, achieving precise incremental update delivery, reducing the amount of data transmitted over the network, and lowering the processing overhead of the configuration relay node and the parsing overhead of the service node, making high-frequency, local configuration updates more efficient. A message middleware is introduced as the channel for configuration update publishing and subscription, and a configuration relay node is established as an intermediate layer. The message middleware decouples the publisher (configuration management end) and the consumer (configuration relay node), possessing the ability to smooth traffic peaks and valleys, thus smoothing out sudden large-scale configuration update traffic and avoiding direct impact on downstream systems. The introduction of the configuration relay node distributes the pressure on the central server to multiple horizontally scalable intermediate nodes, eliminating single-point bottlenecks and alleviating the concurrent pressure faced by the central server and the backend database. Data is simultaneously stored in a persistent storage device and a cache device. The cache device provides high-speed response to query requests from massive service nodes, greatly improving read performance. The persistent storage device ensures reliable data persistence. When the cache misses, the request can be returned to the persistent storage. This two-level storage ensures system stability and high performance.

[0080] This invention also provides a configuration distribution system, including a configuration relay node for executing the configuration data distribution method as described in any embodiment of this invention, a configuration management terminal, a message middleware, and multiple service nodes for providing target services. The process of configuration distribution through interaction between various devices in the system can be referred to the description in the preceding embodiments, and will not be repeated here.

[0081] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing an electronic device according to embodiments of the present invention. Figure 5 The electronic device shown may be a configuration relay node in a configuration distribution system. Figure 5 The example shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0082] like Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage section 508 into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the computer system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0083] The following components are connected to I / O interface 505: input section 506 including keyboard, mouse, etc.; output section 507 including cathode ray tube, liquid crystal display, etc., and speakers, etc.; storage section 508 including hard disk, etc.; and communication section 509 including network interface card, such as modem, etc. Communication section 509 performs communication processing via a network such as the Internet. Drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0084] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by CPU 501, it performs the functions defined above in the system of this invention.

[0085] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, etc., or any suitable combination thereof.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0087] The modules and / or units described in the embodiments of the present invention can be implemented in software or hardware. The described modules and / or units can also be housed in a processor; for example, a processor can be described as including an acquisition module, a storage module, and a distribution module. The names of these modules do not necessarily limit the module itself.

[0088] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: The system retrieves configuration update messages published by the configuration management terminal from the message middleware. Each configuration update message includes update data for at least one configuration data block, which is obtained by splitting the overall configuration data of the target service from the configuration management terminal. The system stores the update data of the configuration data blocks included in the configuration update message into a persistent storage device and a cache device. In response to a configuration retrieval request initiated by the target service node among multiple service nodes, the system retrieves the update data of the configuration data blocks from the cache device or persistent storage device and feeds back the update data of the configuration data blocks to the target service node.

[0089] The technical solution of this invention introduces a configuration data block mechanism and encapsulates only the changed module data in the configuration update message, achieving precise incremental update delivery, reducing the amount of data transmitted over the network, and lowering the processing overhead of the configuration relay node and the parsing overhead of the service node, making high-frequency, local configuration updates more efficient. A message middleware is introduced as the channel for configuration update publishing and subscription, and a configuration relay node is established as an intermediate layer. The message middleware decouples the publisher (configuration management end) and the consumer (configuration relay node), possessing the ability to smooth traffic peaks and valleys, and can smooth sudden large-scale configuration update traffic, avoiding direct impact on downstream systems. The introduction of the configuration relay node distributes the pressure of the central server to multiple horizontally scalable intermediate nodes, eliminating single-point bottlenecks and alleviating the concurrent pressure faced by the central server and the backend database. Data is simultaneously stored in a persistent storage device and a cache device. The cache device provides high-speed response to query requests from massive service nodes, greatly improving read performance. The persistent storage device ensures reliable data persistence. When the cache misses, the request can be returned to the persistent storage. This two-level storage ensures system stability and high performance.

[0090] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the configuration data distribution method provided in any embodiment of this invention.

[0091] In the implementation of a computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​as well as conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0093] It should be noted that the collection, use, storage, sharing, and transfer of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations, and require notification to the user and obtaining the user's consent or authorization. Where applicable, user personal information has undergone de-identification and / or anonymization and / or encryption technical processing. In addition, a corresponding operation entry is provided for the user to choose to agree to or reject the automated decision result; if the user chooses to reject, the process proceeds to the expert decision-making process.

[0094] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for distributing configuration data, characterized in that, The method, applied to a configuration relay node in a configuration distribution system, which also includes a configuration management terminal, a message middleware, and multiple service nodes for providing target services, comprises: The configuration update message published by the configuration management terminal is obtained from the message middleware; wherein, the configuration update message includes update data of at least one configuration data block, and the configuration data block is obtained by the configuration management terminal after splitting the overall configuration data of the target service; The update data of the configuration data block included in the configuration update message is stored in a persistent storage device and a cache device; In response to a configuration retrieval request initiated by a target service node among the plurality of service nodes, the updated data of the configuration data block is retrieved from the cache device or the persistent storage device, and the updated data of the configuration data block is fed back to the target service node.

2. The method according to claim 1, characterized in that, Retrieving configuration update messages published by the configuration management terminal from the message middleware includes: Subscribe to the message topics of the message middleware according to the business dimensions that the configured relay node is responsible for. Listen to the message topic and retrieve the configuration update message published to the message topic.

3. The method according to claim 1, characterized in that, In response to the configuration retrieval request initiated by the target service node, the updated data of the configuration data block is retrieved from the cache device or the persistent storage device, including: In response to the configuration retrieval request initiated by the target service node, first query the cache device to obtain the updated data of the configuration data block; If the cache device does not have updated data for the configuration data block, the persistent storage device is queried to obtain the updated data for the configuration data block.

4. The method according to claim 3, characterized in that, After retrieving the updated data of the configuration data block from the persistent storage device, the process further includes: The updated data of the queried configuration data block is written to the cache device.

5. The method according to claim 3, characterized in that, Querying the persistent storage device to obtain the updated data of the configuration data block includes: Determine whether the query volume of the persistent storage device has reached a preset limit; If the number of queries to the persistent storage device does not reach a preset limit, the updated data of the configuration data block is retrieved by querying the persistent storage device.

6. The method according to claim 1, characterized in that, The updated data of the configuration data block is incremental update data, which is used to indicate the data changes of the latest version of the configuration data block compared to the previous version.

7. The method according to claim 6, characterized in that, The configuration retrieval request is initiated by the target service node after comparing the version identifier of the configuration data block and determining that an incremental update of the configuration data block is required.

8. The method according to claim 1, characterized in that, The configuration retrieval request is initiated by the target service node according to a discrete update cycle, which causes the configuration retrieval requests of the multiple service nodes to be distributed in time.

9. A configuration data distribution device, characterized in that, The device comprises a configuration relay node set up in a configuration distribution system, which also includes a configuration management terminal, a message middleware, and multiple service nodes for providing target services. The acquisition module is used to acquire configuration update messages published by the configuration management terminal from the message middleware; wherein, the configuration update message includes update data of at least one configuration data block, and the configuration data block is obtained by the configuration management terminal after splitting the overall configuration data of the target service; The storage module is used to store the updated data of the configuration data block included in the configuration update message into a persistent storage device and a cache device; The distribution module is used to respond to the configuration retrieval request initiated by the target service node among the plurality of service nodes, retrieve the update data of the configuration data block from the cache device or the persistent storage device, and feed back the update data of the configuration data block to the target service node.

10. A configuration distribution system, characterized in that, It includes a configuration relay node for executing the configuration data distribution method as described in any one of claims 1 to 8, as well as a configuration management terminal, a message middleware, and multiple service nodes for providing target services.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the configuration data distribution method as described in any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the configuration data distribution method as described in any one of claims 1 to 8.

Images