Microservice control method and device
By extracting microservice logs and performance indicator characteristics, determining action strategies and generating configuration files, the problems of database cache and data sharding optimization in the microservice architecture are solved, and efficient data access and resource utilization are achieved.
Patent Information
- Application Number
- CN202411641588.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-04-11
- Estimated Expiration
- 2044-11-18
AI Technical Summary
In the microservice architecture environment, it is difficult for the existing technology to effectively utilize database cache and optimize data sharding, resulting in problems such as rising cache miss rate, increasing data access latency and unbalanced resource allocation.
By extracting the query characteristics of database cache and data shards from the log records generated after calling the microservice, and extracting the performance metric characteristics from the microservice's performance metrics, determining the action strategy, and generating configuration files based on preset parsing rules to regulate the microservice's database cache and data shards.
Adaptive regulation of microservices is realized, data access efficiency is improved, latency is reduced, and resource utilization is optimized, which is suitable for high-performance data processing needs in large-scale distributed microservice environments.
Smart Images

Figure CN119149597B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to database management systems, machine learning, and microservice architecture optimization technologies, and more particularly to microservice control methods and devices thereof. Background Art
[0002] Microservices is an emerging software architecture that splits a large single application and service into dozens of supporting microservices. A microservice strategy can make work easier by scaling individual components rather than the entire application stack to meet service level agreements. Currently, in a microservice architecture environment, database management faces unprecedented challenges due to the highly dynamic and increasingly complex environment.
[0003] In the process of implementing the concept of the present invention, the inventors found that although the microservice management and coordination platform (Istio) provides advanced service discovery, load balancing, traffic management and security control functions for the service grid, and enhances the flexibility and controllability of the microservice architecture, Istio lacks effective utilization of database cache and optimized layout of data shards in terms of service-level governance capabilities. Summary of the invention
[0004] In view of this, the present invention provides a microservice regulation method, apparatus, device, medium and program product.
[0005] One aspect of the present invention provides a microservice regulation method, comprising: extracting a first data query feature for a database cache and a second data query feature for a data shard from a log record generated after a microservice is called; extracting a performance indicator feature from a performance indicator of the microservice; determining an action strategy based on the first data query feature, the second data query feature, and the performance indicator feature, wherein the action strategy is used to characterize an adjustment method for a database cache and / or data shard; parsing the action strategy based on a preset parsing rule that matches a policy type of the action strategy to obtain a parsing result; and generating a configuration file for regulating the microservice based on the parsing result.
[0006] According to an embodiment of the present invention, log records include inter-service call log records and database access log records; based on the first data query characteristics, the second data query characteristics and the performance indicator characteristics, an action strategy is determined, including: based on the first data query characteristics, the second data query characteristics and the performance indicator characteristics, determining an adjustment method for the database cache; and / or based on the second data query characteristics, determining an adjustment method for data sharding.
[0007] According to an embodiment of the present invention, the second data query feature includes the amount of read and write requests and the request response time; based on the second data query feature, determining the adjustment method for the data sharding includes: identifying the data access pattern for the data sharding according to the amount of read and write requests and the request response time, wherein the data access pattern is used to characterize the interaction of reading and writing data; and determining the adjustment method for the data sharding based on the data access pattern.
[0008] According to an embodiment of the present invention, the data shards include multiple data shards, each of the multiple data shards stores different data; based on the data access pattern, determining the adjustment method for the data shards, including: when it is determined that only the first target data shard exists among the multiple data shards, increasing the number of data shards, wherein the data access pattern of the first target data shard represents that the read and write request volume is higher than the first preset request volume, and the request response time is longer than the preset time; when it is determined that only the second target data shard exists among the multiple data shards, reducing the number of data shards, wherein the data access pattern of the second target data shard represents that the read and write request volume is lower than the second preset request volume, and the request response time is shorter than the preset time, and the first preset request volume is higher than the second preset request volume; and when it is determined that the first target data shard and the second target data shard exist at the same time among the multiple data shards, adjusting the shard key of the data shard.
[0009] According to an embodiment of the present invention, determining an adjustment method for data shards based on data access patterns also includes: counting data access patterns of multiple data shards to determine whether there is a situation where the same read / write request is associated with at least two of the multiple data shards; and if it is determined that the same read / write request is associated with at least two of the multiple data shards, readjusting the data in the multiple data shards.
[0010] According to an embodiment of the present invention, determining an adjustment method for data shards based on data access patterns also includes: when it is determined that a first target data shard and a second target data shard exist simultaneously in multiple data shards, adjusting the data in the first target data shard and the second target data shard to obtain a new first target data shard and a new second target data shard, wherein the data access patterns of the new first target data shard and the new second target data shard each indicate that the amount of read and write requests is lower than the first preset request amount, and higher than the second preset request amount.
[0011] According to an embodiment of the present invention, the first data query feature includes: attribute features of data stored in the cache, cache occupancy and cache concurrency features, the second data query feature includes the amount of read and write requests, and the performance indicator feature includes resource usage features; based on the first data query feature, the second data query feature and the performance indicator feature, determining the adjustment method for the database cache, including: adjusting the size of the database cache based on the cache occupancy and resource usage features; implementing a locking mechanism for the database cache according to the cache concurrency features; configuring the cache priority for the database cache according to the amount of read and write requests; adjusting the expiration time of the database cache according to the attribute features of the data stored in the cache; updating the cached data of the database cache when it is determined that the data in the data shard associated with the data stored in the cache has changed; and removing the cached data of the database cache according to the amount of read and write requests associated with the data stored in the cache and the attribute features of the data stored in the cache.
[0012] According to an embodiment of the present invention, the preset parsing rules include a parsing field that matches the policy type; based on the preset parsing rules that match the policy type of the action policy, the action policy is parsed to obtain a parsing result, including: according to the parsing field, the action policy is parsed to obtain a parsing result.
[0013] According to an embodiment of the present invention, an action strategy is determined based on a first data query feature, a second data query feature, and a performance indicator feature, including: constructing a first state vector for a database cache and a second state vector for a data shard according to the first data query feature, the second data query feature, and the performance indicator feature; inputting the first state vector into a deep reinforcement learning model, and outputting an adjustment method for the database cache, wherein the deep reinforcement learning model is pre-trained based on sample log records generated after calling a microservice and sample performance indicators of the microservice; and / or inputting the second state vector into the deep reinforcement learning model, and outputting an adjustment method for the data shard.
[0014] Another aspect of the present invention provides a microservice regulation device, comprising: a first feature extraction module, used to extract a first data query feature for a database cache and a second data query feature for a data shard from a log record generated after calling a microservice; a second feature extraction module, used to extract a performance indicator feature from the performance indicator of the microservice; a strategy determination module, used to determine an action strategy based on the first data query feature, the second data query feature and the performance indicator feature, wherein the action strategy is used to characterize an adjustment method for a database cache and / or data shard; a parsing module, used to parse the action strategy based on a preset parsing rule that matches the strategy type of the action strategy to obtain a parsing result; and a generation module, used to generate a configuration file for regulating the microservice based on the parsing result, so as to regulate the microservice.
[0015] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.
[0016] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.
[0017] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when being executed.
[0018] According to an embodiment of the present invention, by extracting a first data query feature for database cache and a second data query feature for data sharding from log records generated after calling a microservice, and a performance indicator feature extracted from the performance indicators of the microservice, it is possible to determine in real time the adjustment strategy for the database cache and / or data sharding, and convert the policy analysis into a configuration file that Istio can recognize based on the type of policy, thereby achieving adaptive regulation of microservices, improving data access efficiency, reducing latency and optimizing resource utilization, and is particularly suitable for high-performance data processing requirements in large-scale distributed microservice environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0020] Figure 1 An application scenario diagram of a microservice regulation method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.
[0021] Figure 2 A flow chart of a microservice control method according to an embodiment of the present invention is shown.
[0022] Figure 3 A schematic diagram of a method for adjusting database cache according to an embodiment of the present invention is shown.
[0023] Figure 4 A schematic diagram of determining an adjustment method for data sharding according to an embodiment of the present invention is shown.
[0024] Figure 5 A schematic diagram of determining an action strategy according to an embodiment of the present invention is shown.
[0025] Figure 6 A structural block diagram of a microservice control device according to an embodiment of the present invention is shown.
[0026] Figure 7 A block diagram of an electronic device suitable for implementing a microservice regulation method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0027] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.
[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0031] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, invention, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.
[0032] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.
[0033] In the process of implementing the concept of the present invention, the inventors found that traditional database management technology, based on predefined rules or manual configuration, performs well in static or slowly changing systems. However, in a fast-iterating and highly elastic microservice ecosystem, traditional database management technology is difficult to respond in real time to changes in the interaction mode between services, fluctuations in resource demand, and sudden data access peaks, resulting in increased cache miss rates, increased data access latency, and uneven resource allocation. In severe cases, it will also form a performance bottleneck of the system, affecting the user experience and the stability of the overall service.
[0034] Based on this, the present invention provides a microservice regulation method, device, equipment, medium and program product. The method includes: extracting a first data query feature for a database cache and a second data query feature for a data shard from a log record generated after calling a microservice; extracting a performance indicator feature from a performance indicator of a microservice; determining an action strategy based on the first data query feature, the second data query feature and the performance indicator feature, wherein the action strategy is used to characterize the adjustment method for the database cache and / or data shard; parsing the action strategy based on a preset parsing rule matching the strategy type of the action strategy to obtain a parsing result; and generating a configuration file for regulating the microservice based on the parsing result.
[0035] Figure 1 An application scenario diagram of a microservice regulation method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.
[0036] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0037] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).
[0038] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0039] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.
[0040] It should be noted that the microservice regulation method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the microservice regulation device provided in the embodiment of the present invention can generally be set in the server 105. The microservice regulation method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the microservice regulation device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the microservice regulation method provided in the embodiment of the present invention can also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the microservice regulation device provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.
[0041] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0042] Figure 2 The flowchart of the microservice control method according to an embodiment of the present invention is schematically shown.
[0043] like Figure 2 As shown, the microservice control method 200 includes operations S210-S250.
[0044] In operation S210, a first data query feature for a database cache and a second data query feature for a data shard are extracted from a log record generated after a microservice is called.
[0045] In operation S220 , performance indicator features are extracted from the performance indicators of the microservices.
[0046] In operation S230 , an action strategy is determined based on the first data query feature, the second data query feature, and the performance indicator feature.
[0047] In operation S240, the action policy is parsed based on a preset parsing rule that matches the policy type of the action policy to obtain a parsing result.
[0048] In operation S250 , a configuration file for regulating microservices is generated based on the parsing result.
[0049] According to an embodiment of the present invention, log records may include records generated when a service calls a database cache or a data shard, for example, including but not limited to: cache status, data access statistics, and shard details. The cache status may include the current cache hit rate, miss rate, occupancy rate of each cache partition, etc. Data access statistics may include the recent access frequency of each data item or service, access time pattern, etc. For example, it may be a surge in the access frequency of a specific data item in a certain time period, or a change in the response time of certain services under specific conditions. Shard details may include the read and write request volume, response time, and data volume of each data shard. The first data query feature for the database cache and the second data query feature for the data shard may include but not limited to: request frequency, response time, data access volume, data freshness (i.e., data timeliness), and dependencies between data, etc.
[0050] According to an embodiment of the present invention, the performance indicators of microservices may include system load information and service quality indicators. System load information, which may also be referred to as resource usage, may include CPU usage, memory usage, network I / O status, etc. Service quality indicators may include average response time, throughput, etc. Performance indicator features may include, but are not limited to: CPU usage, memory usage, throughput, etc.
[0051] According to an embodiment of the present invention, an action strategy may be used to characterize an adjustment method for database cache and / or data sharding.
[0052] For example, the adjustment method for database cache may include the adjustment method for database cache size, and the database cache size may be adjusted dynamically according to the analysis of cache usage rate and resource usage, etc. The adjustment method for data shards may include the adjustment of the number of data shards, and the number of data shards may be adjusted dynamically according to the analysis of request frequency and response time of data shards, etc.
[0053] According to an embodiment of the present invention, the preset parsing rule may be a rule for indicating parsing, or may be a rule pre-configured according to a historical action policy. The policy type may include two major types of policies for database cache and data sharding adjustment, and the two major types of policies may be classified according to specific adjustment methods, etc. The parsing result is a result that Istio can recognize. For example, the parsing result for the adjustment method of the database cache size may include the amount of cache size adjustment, such as the specific value or percentage of increase or decrease. The parsing result for the adjustment method of the number of data shards may include the identification of the shard, the type of adjustment operation, the amount of adjustment, etc. The microservices may be regulated according to the configuration file generated for regulating the microservices.
[0054] According to an embodiment of the present invention, by extracting a first data query feature for database cache and a second data query feature for data sharding from log records generated after calling a microservice, and a performance indicator feature extracted from the performance indicators of the microservice, it is possible to determine in real time the adjustment strategy for the database cache and / or data sharding, and convert the policy analysis into a configuration file that Istio can recognize based on the type of policy, thereby achieving adaptive regulation of microservices, improving data access efficiency, reducing latency and optimizing resource utilization, and is particularly suitable for high-performance data processing requirements in large-scale distributed microservice environments.
[0055] Reference below Figure 3~Figure 5 , combined with specific embodiments Figure 2 The method shown is further explained.
[0056] According to an embodiment of the present invention, the microservice control method may include the following: Figure 2 In addition to the operations S210 to S250 shown, an operation may also be included: collecting log records and database access records generated after calling the microservice.
[0057] According to an embodiment of the present invention, an integrated distributed tracing system (such as Jaeger or Zipkin) and a log collection tool (such as Fluentd or Logstash) can be used to automatically collect log records and database access records generated after calling microservices.
[0058] According to another embodiment of the present invention, the microservice control method may include the following: Figure 2In addition to the operations S210 to S250 shown, the operation may also include: preprocessing the first data query feature, the second data query feature, and the performance indicator feature.
[0059] According to an embodiment of the present invention, the preprocessing may include normalization processing, which can eliminate the dimension effect and ensure the comparability between different features. In addition, the first data query feature, the second data query feature and the performance indicator feature after preprocessing are analyzed using a sliding window or a time series to capture the change trend of the feature over time.
[0060] According to an embodiment of the present invention, the logging may include inter-service call logging and database access logging.
[0061] For the above Figure 2 The operation S230 shown determines the action strategy based on the first data query characteristic, the second data query characteristic and the performance indicator characteristic, and may include the operations of: determining the adjustment method for the database cache based on the first data query characteristic, the second data query characteristic and the performance indicator characteristic; and / or determining the adjustment method for the data sharding based on the second data query characteristic.
[0062] According to an embodiment of the present invention, the adjustment method for the database cache may include at least one of the following: adjusting the size of the database cache, implementing a lock mechanism, configuring cache priority, adjusting expiration time, updating cache data, removing cache data, etc.
[0063] According to an embodiment of the present invention, the adjustment method for data sharding may include at least one of the following: adjusting the number of data shards, adjusting sharding keys, and adjusting data.
[0064] Figure 3 A schematic diagram of a method for adjusting database cache according to an embodiment of the present invention is shown.
[0065] According to an embodiment of the present invention, the first data query feature may include: attribute features of data stored in the cache, cache occupancy, and cache concurrency features, the second data query feature may include the amount of read and write requests, and the performance indicator feature may include resource usage features. The attribute features of data stored in the cache may be, for example, features such as data timeliness. The cache concurrency feature is used to characterize high concurrency features in high concurrency scenarios.
[0066] like Figure 3 As shown, the adjustment method for the database cache may include adjusting the size of the database cache, implementing a lock mechanism, configuring cache priority, adjusting the expiration time, updating cache data, and removing cache data.
[0067] According to an embodiment of the present invention, the size of the database cache can be adjusted based on the cache occupancy rate and resource usage characteristics. For example, the size of a specific cache area can be dynamically increased or decreased based on the cache occupancy rate and resource usage characteristics to balance memory usage and performance improvement.
[0068] According to an embodiment of the present invention, a lock mechanism is implemented on the database cache based on cache concurrency characteristics.
[0069] According to an embodiment of the present invention, the lock mechanism includes an optimistic lock or a pessimistic lock mechanism. According to the cache concurrency characteristics, it can be determined whether it is in a high concurrency scenario. In a high concurrency scenario, the atomicity and consistency of the cache update operation can be ensured by implementing an optimistic lock or a pessimistic lock mechanism to avoid data conflicts.
[0070] According to an embodiment of the present invention, a cache priority is configured for a database cache according to the amount of read and write requests.
[0071] According to an embodiment of the present invention, configuring cache priority for a database cache based on the amount of read and write requests may include: when it is determined that the amount of read and write requests is higher than a first preset request amount, configuring priority cache instructions for data associated with the amount of read and write requests so that it is preferentially stored in the database cache.
[0072] According to an embodiment of the present invention, configuring cache priority for database cache may also include: obtaining request frequency, response time, data access volume, data freshness, and access time pattern. For example, the request frequency may be the number of times each data item or service is requested, or the number of requests per unit time. The response time may be the average and distributed response time for data access or service call. The data access volume may be the total amount of data read or modified within a period of time. The data freshness may be the length of time since the last update of the data, which measures the timeliness of the data. The access time pattern may be a time series of data access, reflecting the periodicity of access or the peak of access in a specific period of time. The request frequency, response time, data access volume, data freshness, and access time pattern are input into the time series prediction model, and with the help of the clustering algorithm, future hot data is identified and predicted, cache resource allocation is dynamically adjusted, high-frequency access data is cached first, and fast access to hot data is ensured.
[0073] According to an embodiment of the present invention, the expiration time of the database cache is adjusted according to the attribute characteristics of the data stored in the cache. For example, a shorter cache expiration time can be set for data with a timeliness higher than a preset timeliness, while a longer expiration time can be set for relatively static data to reduce the waste of resources caused by invalid cache.
[0074] According to an embodiment of the present invention, when it is determined that data in a data shard associated with data stored in a cache has changed, cache data in a database cache is updated. For example, according to a data change event in a data shard associated with data stored in a cache, the cache may be synchronized in real time, or a more efficient comparison algorithm may be used to reduce synchronization overhead to ensure data consistency.
[0075] According to an embodiment of the present invention, cached data in a database cache is removed according to the amount of read and write requests associated with the data stored in the cache and the attribute characteristics of the data stored in the cache. For example, the cached data in the cache may be divided into data importance according to the amount of read and write requests associated with the data stored in the cache and the attribute characteristics of the data stored in the cache, and the elimination strategy may be dynamically adjusted according to the data importance to avoid non-optimal removal of critical data.
[0076] According to an embodiment of the present invention, by determining the adjustment method for the database cache, a fine-grained dynamic cache management mechanism can be implemented to effectively improve data access speed, reduce latency, and ensure data consistency and system stability.
[0077] Figure 4 A schematic diagram of determining an adjustment method for data sharding according to an embodiment of the present invention is shown.
[0078] According to an embodiment of the present invention, the second data query feature may include the read and write request volume and the request response time.
[0079] like Figure 4 As shown, determining the adjustment method for data sharding based on the second data query feature may include the following operations: identifying a data access pattern 403 for the data sharding based on the read / write request volume 401 and the request response time 402. Based on the data access pattern 403, determining the adjustment method 404 for the data sharding.
[0080] According to an embodiment of the present invention, data access mode 403 can be used to characterize the interactive situation of reading and writing data. The interactive situation of reading and writing data may include, for example, the situation that the amount of read and write requests is higher than the first preset request amount, and the request response time is longer than the preset time, or the situation that the amount of read and write requests is lower than the second preset request amount, and the request response time is shorter than the preset time, or the situation that the amount of read and write requests is higher than the second preset request amount and lower than the first preset request amount, and the request response time is shorter than the preset time, etc.
[0081] According to an embodiment of the present invention, a real-time monitoring tool (such as Prometheus) may be integrated to continuously track the read and write request volume 401 and request response duration 402 of the data shards to ensure accurate monitoring of the data.
[0082] According to an embodiment of the present invention, the data sharding adjustment method 404 may include but is not limited to dynamically increasing or decreasing the number of shards, adjusting the sharding key to optimize data locality, and reallocating data to different shards to balance the load.
[0083] According to an embodiment of the present invention, the adjustment method for data sharding is determined, which can realize intelligent identification of data access patterns and automatic generation of dynamic sharding strategies, thereby ensuring efficient and stable operation of the database under the Istio microservice architecture.
[0084] According to an embodiment of the present invention, the data shards may include multiple data shards, and each of the multiple data shards stores different data; based on the data access mode, determining the adjustment method for the data shards may include the following operations: when it is determined that only the first target data shard exists among the multiple data shards, increasing the number of data shards. When it is determined that only the second target data shard exists among the multiple data shards, reducing the number of data shards. When it is determined that both the first target data shard and the second target data shard exist among the multiple data shards, adjusting the sharding key of the data shard.
[0085] According to an embodiment of the present invention, the data access pattern of the first target data shard indicates that the amount of read and write requests is higher than the first preset amount of requests, and the request response time is longer than the preset time, which can indicate that the first target data shard is under excessive load pressure, for example, a small portion of data is frequently accessed.
[0086] According to an embodiment of the present invention, the data access pattern of the second target data shard indicates that the read and write request amount is lower than the second preset request amount, and the request response time is less than the preset time. The first preset request amount is higher than the second preset request amount. It can indicate that the second target data shard may be too subdivided, resulting in a waste of resources. For example, some data becomes less active over time, or business requirements change, etc.
[0087] The first preset request amount may be, for example, a request amount considered to be abnormally high during the actual data read and write interaction process. The second preset request amount may be, for example, a request amount considered to be abnormally low during the actual data read and write interaction process.
[0088] According to an embodiment of the present invention, when it is determined that the first target data shard and the second target data shard exist simultaneously in multiple data shards, it can be shown that a specific sharding key causes uneven data distribution, data access to certain data shards is too concentrated, while other data shards are relatively idle. For example, a performance bottleneck occurs in a specific query mode.
[0089] According to the embodiments of the present invention, a data synchronization mechanism is adopted between data shards to ensure that the latest state of data is retained during the process of adding or reducing data shards, and technologies such as distributed transaction protocols or two-phase commit are used to solve data conflict problems. Data verification tasks can also be performed periodically to check data integrity through verification codes or other verification mechanisms, and potential data inconsistencies caused by network delays, hardware failures, etc. can be discovered and automatically repaired.
[0090] According to an embodiment of the present invention, by increasing the number of data shards, frequently accessed data can be dispersed, and by dividing the data more evenly onto more shards, the pressure on a single shard can be reduced, thereby improving the processing power and response speed of the overall system. By merging these low-load data shards and reducing the total number of data shards, resource allocation can be optimized, maintenance overhead can be reduced, and data management complexity can be simplified. By adjusting the sharding key of the data shards to make the data distribution more balanced, the query efficiency and the load balancing of the overall system can be improved. For example, a new sharding key that better represents the data access pattern can be selected so that hot data can be more evenly distributed across the shards.
[0091] According to an embodiment of the present invention, considering that with business development or changes in data access patterns, the original data distribution is no longer optimal, resulting in poor data locality and frequent cross-data shard queries.
[0092] Based on this, based on the data access pattern, determining the adjustment method for the data shards can also include operations: counting the data access patterns of multiple data shards, determining whether there is a situation where the same read / write request is associated with at least two of the multiple data shards; and if it is determined that the same read / write request is associated with at least two of the multiple data shards, readjusting the data in the multiple data shards.
[0093] According to an embodiment of the present invention, by counting the data access patterns of multiple data shards, when it is determined that the same read and write request is associated with at least two data shards among the multiple data shards, the data distribution is re-planned, and data that are frequently accessed together are placed on the same shard or adjacent shards as much as possible, thereby reducing cross-shard queries and improving data access efficiency.
[0094] According to an embodiment of the present invention, considering that with the development of business or changes in data access patterns, the original data distribution is no longer optimal, resulting in huge differences in data access frequencies between certain data shards and unbalanced load.
[0095] Based on this, based on the data access pattern, determining the adjustment method for the data shards may also include the following operations: when it is determined that the first target data shard and the second target data shard exist simultaneously in multiple data shards, adjusting the data in the first target data shard and the second target data shard to obtain a new first target data shard and a new second target data shard, wherein the data access patterns of the new first target data shard and the new second target data shard each indicate that the read and write request amount is lower than the first preset request amount and higher than the second preset request amount.
[0096] According to an embodiment of the present invention, by adjusting the data in the first target data shard and the second target data shard, the data can be reallocated to different shards to balance the load.
[0097] Figure 5 A schematic diagram of determining an action strategy according to an embodiment of the present invention is shown.
[0098] like Figure 5 As shown, in this embodiment, the action strategy is determined based on the first data query feature, the second data query feature and the performance indicator feature, and the operations may also be included: constructing a first state vector 504 for the database cache and a second state vector 505 for the data shard according to the first data query feature 501, the second data query feature 502 and the performance indicator feature 503; inputting the first state vector 504 into the deep reinforcement learning model M, and outputting an adjustment method 506 for the database cache, wherein the deep reinforcement learning model M is pre-trained based on the sample log records generated after calling the microservice and the sample performance indicators of the microservice; and / or inputting the second state vector 505 into the deep reinforcement learning model M, and outputting an adjustment method 507 for the data shard.
[0099] According to an embodiment of the present invention, the algorithm framework of the deep reinforcement learning model M can choose to use Proximal Policy Optimization (PPO) as the core algorithm framework, because it performs well in balancing exploration and utilization, learning stability and sample efficiency, and is suitable for handling complex decision-making problems of database management under a microservice architecture. PPO avoids a large deviation from the old strategy by limiting the strategy update step size using the clip function, thereby achieving stable learning performance on the continuous action strategy.
[0100] The network architecture of the deep reinforcement learning model M may include an input layer, a hidden layer, and an output layer. The input layer may receive a state vector whose dimension corresponds to the number of features in the state representation. The hidden layer may have at least two fully connected layers, using ReLU as an activation function to enhance the nonlinear expression capability of the model. The output layer may set the number of output nodes according to the dimension of the action strategy, using a softmax function for discrete action strategies or a tanh function for continuous action strategies to ensure the probability distribution of the output action.
[0101] Sample log records generated after calling a microservice within a period of time and sample performance indicators of the microservice can be obtained. A first sample data query feature for database cache and a second sample data query feature and sample performance indicator feature for data sharding are extracted from the sample log records and sample performance indicator features of the microservice respectively. A first sample state vector for database cache and a second sample state vector for data sharding are constructed based on the first sample data query feature, the second sample data query feature and the sample performance indicator feature.
[0102] An action strategy that matches the first sample state vector and an action strategy that matches the second sample state vector can be defined as the model learning output target. A multi-dimensional reward function can be designed to reflect the comprehensive improvement of system performance. For example, a multi-dimensional reward function may include but is not limited to: response time reward, throughput reward, resource utilization reward, penalty mechanism, etc. A weighted sum method can be used to combine various rewards and penalties to ensure that the model strikes a balance between multiple objectives.
[0103] For example, response time rewards can be positive rewards based on the reduction in request response time, encouraging latency reduction. Throughput rewards can be rewards given when system throughput increases, encouraging increased processing power. Resource utilization rewards can be rewards for efficient use of resources (such as CPU, memory), promoting economic allocation of resources. Penalty mechanisms can be negative penalties for cache misses, data inconsistencies, or system instability to avoid bad decisions.
[0104] The first sample state vector and the second sample state vector can be input into the initially constructed deep reinforcement learning model M, and the sample adjustment method for the database cache and the sample adjustment method for the data sharding can be output. Based on the sample adjustment method for the database cache and the sample adjustment method for the data sharding, a sample configuration file for regulating the microservice is generated, the microservice is regulated, and the reward value is determined. The first sample state vector, the second sample state vector, the sample adjustment method for the database cache, the sample adjustment method for the data sharding, and the reward value are matched and stored in the experience playback buffer for batch training to reduce variance and improve learning efficiency. Calculating the cumulative reward for n steps helps the model learn long-term benefits more effectively. Using methods such as grid search or Bayesian optimization, find the optimal hyperparameter combination, including learning rate, discount factor, clip range, etc., to accelerate convergence and improve performance.
[0105] You can use techniques such as Asynchronous Advantage Executor (A3C) or Vectorized environments in machine learning to speed up the training process. You can save the trained model regularly and perform offline verification before launching the new action strategy to ensure the robustness of the strategy.
[0106] According to an embodiment of the present invention, constructing a deep reinforcement learning model in the above manner can perform efficient and adaptive management of database cache and / or data sharding under the Istio microservice architecture, thereby significantly improving system performance and resource utilization efficiency.
[0107] According to an embodiment of the present invention, real-time monitoring and alarming can be performed for microservices. For example, a monitoring system (such as Prometheus) can be integrated to track key performance indicators such as cache hit rate, miss rate, cache usage rate, system response time, etc. in real time, and an alarm can be triggered immediately once an abnormality is found.
[0108] According to one embodiment of the present invention, an action strategy can be evaluated. For example, the effect of the adjustment method for the database cache can be evaluated regularly or in real time, including changes in cache hit rate, improvement in response time, resource utilization, etc., and the evaluation results can be fed back to the deep reinforcement learning model as learning input to promote iterative optimization of the model. An evaluation mechanism for the effect of data sharding strategy can be constructed, and performance indicators (such as query response time, throughput, etc.) after data sharding adjustment can be analyzed regularly or in real time, and compared with the data before adjustment, the results of strategy optimization can be quantified to obtain evaluation results.
[0109] According to one embodiment of the present invention, retraining of the deep reinforcement learning model can be automatically triggered based on real-time monitoring data and evaluation results, and the action strategy can be quickly adjusted to cope with dynamic changes in system access patterns or resource requirements.
[0110] According to one embodiment of the present invention, anomaly detection logic can be integrated to immediately trigger an emergency response process, including rolling back to the previous stable strategy or starting a quick recovery program, to ensure service continuity, once an abnormal situation such as data inconsistency, shard failure, or a sharp drop in system performance is discovered.
[0111] According to an embodiment of the present invention, the preset parsing rule may include a parsing field that matches the policy type.
[0112] For the above Figure 2 The operation S240 shown, parsing the action policy based on a preset parsing rule matching the policy type of the action policy to obtain a parsing result, may include the operation of parsing the action policy according to a parsing field to obtain a parsing result.
[0113] According to an embodiment of the present invention, the size adjustment method for the database cache may include increasing / decreasing the cache size of a specific data item. The target data item identifier and the parsing result of the amount of cache size adjustment may be parsed from the action strategy for increasing / decreasing the cache size of the specific data item. The change adjustment method for the database cache may include adjusting the expiration time of the database cache, implementing a lock mechanism for the database cache, configuring a cache priority for the database cache, updating the cache data of the database cache, and removing the cache data of the database cache. The data item identifier of the change action strategy and the new action strategy type may be parsed from the action strategy for changing the database cache.
[0114] According to an embodiment of the present invention, the adjustment method for data sharding may include increasing or decreasing the number of data shards. The identifier (such as table name) of the data shard, the operation type (increase or decrease) and the number of parsed results may be obtained from the action strategy for increasing or decreasing the number of data shards.
[0115] According to an embodiment of the present invention, the adjustment method for data sharding may include re-adjusting the data in multiple data shards. That is, reallocating the data to different data shards. The analysis results of which data (possibly based on certain keys or ranges) should be reallocated to which new shards can be obtained from the action strategy for re-adjusting the data in multiple data shards. The analysis fields may include source data shard information, target data shard information, and reallocation conditions or rules.
[0116] According to an embodiment of the present invention, the adjustment method for data sharding may include adjusting the sharding key to optimize data locality. The affected data sharding, current and new sharding key definitions may be parsed from the action strategy for adjusting the sharding key.
[0117] According to an embodiment of the present invention, the action policy can also be used to characterize the adjustment method for resource allocation. The adjustment method for resource allocation can include dynamically adjusting the resource quota of the service instance. The service instance identifier, the adjusted resource type (CPU, memory, etc.) and the adjusted amount can be parsed from the action policy for dynamically adjusting the resource quota of the service instance.
[0118] According to an embodiment of the present invention, by parsing the action policy through parsing fields, a parsing result that can be recognized by Istio can be obtained, which is conducive to generating a configuration file for regulating microservices and realizing uninterrupted action policy application.
[0119] According to an embodiment of the present invention, by generating a configuration file for regulating microservices, database cache configuration automation and data sharding configuration automation can be achieved at the Istio service level. For example, the configuration file of the cache system (such as Redis, Memcached) can be dynamically adjusted according to the configuration file for regulating microservices. After determining the data sharding adjustment method, the sharding structure adjustment is seamlessly performed in the runtime environment according to the configuration file for regulating microservices, ensuring the continuity and availability of services during data migration and minimizing the impact on the business.
[0120] Figure 6 A structural block diagram of a microservice control device according to an embodiment of the present invention is shown.
[0121] like Figure 6 As shown, the microservice control device 600 includes a first feature extraction module 610, a second feature extraction module 620, a strategy determination module 630, a parsing module 640 and a generating module 650.
[0122] The first feature extraction module 610 is used to extract a first data query feature for the database cache and a second data query feature for the data shard from the log record generated after the microservice is called.
[0123] The second feature extraction module 620 is used to extract performance indicator features from the performance indicators of the microservices.
[0124] The strategy determination module 630 is used to determine an action strategy based on the first data query feature, the second data query feature and the performance indicator feature, wherein the action strategy is used to characterize the adjustment method of the database cache and / or data sharding.
[0125] The parsing module 640 is used to parse the action policy based on a preset parsing rule that matches the policy type of the action policy to obtain a parsing result.
[0126] The generation module 650 is used to generate a configuration file for regulating the microservice based on the analysis result.
[0127] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.
[0128] For example, any of the first feature extraction module 610, the second feature extraction module 620, the strategy determination module 630, the parsing module 640, and the generation module 650 can be combined in one module / unit / sub-unit for implementation, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the first feature extraction module 610, the second feature extraction module 620, the strategy determination module 630, the parsing module 640, and the generation module 650 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the first feature extraction module 610, the second feature extraction module 620, the strategy determination module 630, the parsing module 640 and the generation module 650 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.
[0129] It should be noted that the microservice control device part in the embodiment of the present invention corresponds to the microservice control method part in the embodiment of the present invention. The description of the microservice control device part specifically refers to the microservice control method part, which will not be repeated here.
[0130] Figure 7 A block diagram of an electronic device suitable for implementing a microservice regulation method according to an embodiment of the present invention is shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0131] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.
[0132] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.
[0133] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.
[0134] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.
[0135] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.
[0136] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.
[0137] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .
[0138] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the microservice regulation method provided by the embodiment of the present invention.
[0139] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0140] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0141] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.
[0143] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.
Claims
1. A microservice control method, characterized in that: The method comprises: Extracting a first data query feature for the database cache and a second data query feature for the data shard from the log records generated after calling the microservice, wherein the first data query feature includes: attribute features of the data stored in the cache, cache occupancy, and cache concurrency features, and the second data query feature includes the read and write request volume and request response time; Extracting performance indicator features from the performance indicators of the microservice, wherein the performance indicator features include resource usage features; Determine an action strategy based on the first data query feature, the second data query feature, and the performance indicator feature, wherein the action strategy is used to characterize an adjustment method for the database cache and / or the data sharding; Parsing the action strategy based on a preset parsing rule that matches the strategy type of the action strategy to obtain a parsing result; and Based on the analysis result, a configuration file for regulating the microservice is generated.
2. The method according to claim 1, characterized in that: The log records include inter-service call log records and database access log records; The determining of an action strategy based on the first data query feature, the second data query feature, and the performance indicator feature includes: Determining an adjustment method for the database cache based on the first data query characteristic, the second data query characteristic, and the performance indicator characteristic; and / or Based on the second data query feature, determine an adjustment method for the data shard.
3. The method according to claim 2, characterized in that The determining, based on the second data query feature, a way to adjust the data shards includes: Identify a data access pattern for the data shard according to the read and write request volume and the request response duration, wherein the data access pattern is used to characterize an interaction situation of reading and writing data; and Based on the data access pattern, determine how to adjust the data shards.
4. The method according to claim 3, characterized in that The data shards include a plurality of data shards, and each of the plurality of data shards stores different data; The determining, based on the data access mode, a way to adjust the data shards includes: In the case of determining that only the first target data shard exists among the plurality of data shards, increasing the number of the data shards, wherein the data access mode of the first target data shard indicates that the read and write request amount is higher than the first preset request amount, and the request response time is longer than the preset time; In the case where it is determined that only the second target data shard exists among the multiple data shards, reducing the number of the data shards, wherein the data access pattern of the second target data shard indicates that the read and write request amount is lower than the second preset request amount, and the request response time is shorter than the preset time, and the first preset request amount is higher than the second preset request amount; and When it is determined that the first target data shard and the second target data shard exist simultaneously in the plurality of data shards, the shard keys of the data shards are adjusted.
5. The method according to claim 4, characterized in that The step of determining the adjustment method for the data sharding based on the data access mode further includes: Counting the data access patterns of the plurality of data shards to determine whether the same read / write request is associated with at least two of the plurality of data shards; and In a case where it is determined that the same read / write request is associated with at least two of the multiple data slices, data in the multiple data slices are realigned.
6. The method according to claim 4 or 5, characterized in that: The step of determining the adjustment method for the data sharding based on the data access mode further includes: When it is determined that the first target data shard and the second target data shard exist simultaneously in the plurality of data shards, the data in the first target data shard and the second target data shard are adjusted to obtain a new first target data shard and a new second target data shard, wherein the data access modes of the new first target data shard and the new second target data shard respectively indicate that the read and write request amount is lower than the first preset request amount and higher than the second preset request amount.
7. The method according to claim 2, characterized in that: The determining, based on the first data query feature, the second data query feature, and the performance indicator feature, a method for adjusting the database cache includes: Adjusting the size of the database cache based on the cache occupancy rate and the resource usage characteristics; Implementing a lock mechanism on the database cache according to the cache concurrency characteristics; According to the read and write request amount, configuring cache priority for the database cache; Adjusting the expiration time of the database cache according to the attribute characteristics of the data stored in the cache; When it is determined that data in the data shard associated with the data stored in the cache has changed, updating cache data in the database cache; and The cached data in the database cache is removed according to the read and write request amount associated with the data stored in the cache and the attribute characteristics of the data stored in the cache.
8. The method according to claim 1, characterized in that The preset parsing rule includes a parsing field matching the policy type; The step of parsing the action strategy based on a preset parsing rule that matches the strategy type of the action strategy to obtain a parsing result includes: The action strategy is parsed according to the parsing field to obtain the parsing result.
9. The method according to claim 1, characterized in that: The determining of an action strategy based on the first data query feature, the second data query feature, and the performance indicator feature includes: Constructing a first state vector for the database cache and a second state vector for the data shard according to the first data query feature, the second data query feature, and the performance indicator feature; Inputting the first state vector into a deep reinforcement learning model, and outputting an adjustment method for the database cache, wherein the deep reinforcement learning model is pre-trained based on sample log records generated after calling the microservice and sample performance indicators of the microservice; and / or The second state vector is input into the deep reinforcement learning model, and an adjustment method for the data shard is output.
10. A microservice control device, characterized in that: The device comprises: A first feature extraction module is used to extract a first data query feature for the database cache and a second data query feature for the data shard from the log records generated after calling the microservice, wherein the first data query feature includes: attribute features of the data stored in the cache, cache occupancy rate and cache concurrency features, and the second data query feature includes the read and write request volume and request response time; A second feature extraction module is used to extract performance indicator features from the performance indicators of the microservice, wherein the performance indicator features include resource usage features; A strategy determination module, used to determine an action strategy based on the first data query feature, the second data query feature and the performance indicator feature, wherein the action strategy is used to characterize an adjustment method for the database cache and / or the data sharding; a parsing module, configured to parse the action strategy based on a preset parsing rule matching the strategy type of the action strategy to obtain a parsing result; and A generation module is used to generate a configuration file for regulating the microservice based on the analysis result, so as to regulate the microservice.
Citation Information
Patent Citations
Modular data management system based on multi-semantic expression
CN114327951A
Data query method under micro-service architecture
CN117633010A