Cluster service optimization method and device

Abstract

The invention provides a cluster service optimization method and device.The method comprises the steps that request information of a query request for a target distributed service cluster and cluster index information of the target distributed service cluster are obtained, and the request information comprises a request body domain specific language DSL statement of the query request; performing statement early-warning analysis according to the request body DSL statement to generate first early-warning data, performing structure early-warning analysis according to the target index structure and the request body DSL statement to generate second early-warning data, and respectively obtaining optimization information of a preset statement generation code and optimization information of the target index structure according to the first early-warning data and the second early-warning data, and optimizing the preset statement generation code and the target index structure. Therefore, before the cluster goes wrong, the statement generation code and the index structure are optimized in advance through an early warning inspection mode, and the capability of cluster performance management is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a cluster service optimization method and device. Background Art

[0002] Elasticsearch is a distributed search and analysis engine based on open source technology. It can process massive amounts of data, support multiple search types such as full-text search, structured search, and combined search, and can achieve real-time data processing under a distributed architecture.

[0003] In related technologies, when jitter occurs in the Elasticsearch cluster, the only way to locate the problem is on-site. If it is confirmed that the cluster jitter is caused by the business side, the corresponding business manager is found based on the problem index and a recommended optimization plan is given. The business manager evaluates the optimization time and solves the problem based on the on-site status and the problem optimization plan.

[0004] However, if optimization analysis is performed after cluster jitter occurs, the cluster jitter will continue to affect the business during the optimization period, aggravating the impact of the jitter. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a cluster service optimization method and device to solve the problem that optimization analysis is performed after cluster jitter occurs, resulting in the cluster jitter continuing to affect the service during optimization and aggravating the impact of the jitter.

[0006] In a first aspect, an embodiment of the present application provides a cluster service optimization method, including:

[0007] Obtaining request information of a query request for a target distributed business cluster and cluster index information of the target distributed business cluster, the request information comprising: a request body domain specific language (DSL) statement of the query request, the request body DSL statement being a statement generated by using a preset statement generation code, and the cluster index information comprising: a target index structure of the target distributed business cluster;

[0008] Perform statement warning analysis according to the request body DSL statement to generate first warning data;

[0009] Performing structural warning analysis according to the target index structure and the request body DSL statement to generate second warning data;

[0010] According to the first warning data and the second warning data, optimization information of the preset statement generation code and optimization information of the target index structure are respectively obtained to optimize the preset statement generation code and the target index structure.

[0011] In an optional implementation manner, performing statement warning analysis according to the request body DSL statement to generate first warning data includes:

[0012] Determining a target query type and a paging depth of the query request according to the request body DSL statement;

[0013] If the target query type is a fuzzy query type and / or the paging depth exceeds a preset depth threshold, generating the first warning data.

[0014] In an optional implementation manner, performing structure warning analysis according to the target index structure and the request body DSL statement to generate second warning data includes:

[0015] If the target index structure is an object array index structure or a join index structure, generating the second warning data; and / or,

[0016] Judging whether the target query type of the query request and the target index structure match according to a preset matching relationship between the query type and the index structure;

[0017] If the target query type and the target index structure do not match, generating the second warning data.

[0018] In an optional implementation manner, the cluster index information further includes: the number of nodes, the number of shards, and index data of the target distributed service cluster; the method further includes:

[0019] Performing shard health warning analysis according to the number of nodes, the number of shards, and the index data to obtain third warning data;

[0020] Performing data volume warning analysis according to the index data to generate fourth warning data;

[0021] According to the third warning data and the fourth warning data, respectively obtaining optimization information of the number of shards and optimization information of the index data to optimize the number of shards and the index data.

[0022] In an optional implementation manner, the method further includes:

[0023] Obtaining a bottom-layer query request of the request body DSL statement in the target distributed service cluster according to the target index structure and the request body DSL statement;

[0024] If the bottom-layer query request is a range query request, generating fifth warning data;

[0025] Obtain the optimization information of the preset statement generation code and / or the optimization information of the target index structure according to the fifth warning data.

[0026] In an alternative embodiment, the method further includes:

[0027] If the underlying query request is a minimum matching query request and the target distributed service cluster is an updated service cluster, generate sixth warning data;

[0028] Obtain the optimization information of the preset statement generation code and / or the optimization information of the target index structure according to the sixth warning data.

[0029] In an alternative embodiment, the request information further includes: a request index corresponding to the query request, where the request index is used to indicate a target index resource in the target distributed service cluster, and the method further includes:

[0030] Calculate a query rate for the target index resource according to the request index;

[0031] If the query rate exceeds a preset query rate threshold, generate seventh warning data;

[0032] Obtain the optimization information of the target index resource according to the seventh warning data to optimize access to the target index resource.

[0033] In an alternative embodiment, the calculating a query rate for the target index resource according to the request index includes:

[0034] According to the target query type of the query request, replace the values in the request body DSL statement with placeholders corresponding to the target query type to obtain a target request body DSL signature;

[0035] Calculate the query rate under the target request body DSL signature according to the request index.

[0036] In an alternative embodiment, the request information further includes: a request link identifier of the query request; the method further includes:

[0037] Obtain the service application corresponding to the query request according to the request link identifier of the query request;

[0038] Push the optimization information of the preset statement generation code and the optimization information of the target index structure to the application maintainer of the service application corresponding to the query request.

[0039] In a second aspect, an embodiment of the present application further provides a cluster service optimization device, including:

[0040] An acquisition module, configured to acquire a query request for a target distributed service cluster and cluster index information of the target distributed service cluster, where the cluster index information includes: a target index structure of the target distributed service cluster;

[0041] A processing module, configured to generate code using a preset request body domain-specific language (DSL) statement to generate request information for the query request, where the request information includes: the request body domain-specific language (DSL) statement of the query request;

[0042] The processing module is further configured to perform statement warning analysis based on the request body DSL statement to generate first warning data;

[0043] The processing module is further configured to perform structure warning analysis based on the target index structure and the request body DSL statement to generate second warning data;

[0044] The acquisition module is configured to respectively acquire optimization information for the preset statement generation code and optimization information for the target index structure according to the first warning data and the second warning data, so as to optimize the preset statement generation code and the target index structure.

[0045] This application provides a cluster service optimization method and apparatus. The method includes: acquiring request information for a query request for a target distributed service cluster and cluster index information of the target distributed service cluster, where the request information includes: the request body domain-specific language (DSL) statement of the query request, performing statement warning analysis based on the request body DSL statement to generate first warning data, performing structure warning analysis based on the target index structure and the request body DSL statement to generate second warning data, and respectively acquiring optimization information for the preset statement generation code and optimization information for the target index structure according to the first warning data and the second warning data, so as to optimize the preset statement generation code and the target index structure. Thus, before a problem occurs in the cluster, the statement generation code and the index structure are pre-optimized through a warning inspection method, improving the ability of cluster performance management. Description of the Drawings

[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 A schematic diagram of the functional modules for cluster service optimization provided by the embodiments of this application;

[0048] Figure 2 Flow schematic diagram of the cluster service optimization method provided by the embodiment of the present application Figure 1 ;

[0049] Figure 3 Flow schematic diagram of the cluster service optimization method provided by the embodiment of the present application Figure 2 ;

[0050] Figure 4 Flow schematic diagram of the cluster service optimization method provided by the embodiment of the present application Figure 3 ;

[0051] Figure 5 Flow schematic diagram of the cluster service optimization method provided by the embodiment of the present application Figure 4 ;

[0052] Figure 6 Flow schematic diagram of the cluster service optimization method provided by the embodiment of the present application Figure 5 ;

[0053] Figure 7 Flow schematic diagram of the cluster service optimization method provided by the embodiment of the present application Figure 6 ;

[0054] Figure 8 Flow schematic diagram of the cluster service optimization method provided by the embodiment of the present application Figure 7 ;

[0055] Figure 9 Structure schematic diagram of the cluster service optimization device provided by the embodiment of the present application;

[0056] Figure 10 Structure schematic diagram of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0057] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part rather than all of the embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0058] First, the professional terms related to the present application are described:

[0059] Elasticsearch, abbreviated as ES, is a distributed search and analysis engine based on open-source technology. It can handle massive amounts of data, support various search types such as full-text search, structured search, and combined search, and can achieve real-time data processing in a distributed architecture. Elasticsearch uses an inverted index to accelerate search operations, and this indexing method is very suitable for full-text retrieval. At the same time, Elasticsearch provides powerful aggregation capabilities, making it outstanding in analyzing complex data patterns and trends. Its scalability and flexible RESTful API (Representational State Transfer API) enable it to be widely used in various scenarios such as log analysis, real-time search, data analysis, and monitoring. Among them, the RESTful API is an application programming interface that follows the REST architectural specification and supports interaction with RESTful web services.

[0060] Cluster jitter: refers to the fluctuation of the load or performance of the cluster, which may be caused by the hardware environment or business traffic. For example, when the read and write traffic of a certain business index suddenly increases, a large number of random I / Os and CPU occupations are generated, squeezing the cluster resources, resulting in other requests being placed in the queue or even rejected. When there is cluster jitter, at the lightest level, the response time (RT) of processing requests increases significantly, and at the heaviest level, requests fail and the cluster crashes.

[0061] In related technologies, when the Elasticsearch cluster experiences jitter, only the problem can be located on-site. If it is confirmed that the cluster jitter is caused by the business side, then the corresponding business responsible person is found based on the problem index, and a proposed optimization plan is given. The business responsible person evaluates the optimization time and solves the problem in combination with the on-site status and the problem optimization plan. In this solution, when the cluster experiences jitter, the impact on all businesses in the cluster is fatal. At the lightest level, some read and write operations time out, and at the heaviest level, the cluster crashes and becomes completely unavailable. At this time, it is also necessary to temporarily analyze the cause of the jitter based on data such as cluster monitoring. During the analysis period, the cluster jitter will continue to affect the business, exacerbating the impact brought by the jitter. Moreover, even if the cause of the cluster jitter is located and an optimization plan is given, the actual implementation time of the optimization plan is not available, ranging from a few minutes to several days. Before the optimization plan is implemented, the jitter and the risk of jitter still exist. In addition, waiting until the cluster experiences jitter to intervene in troubleshooting and optimization, even if the cluster is not jittering at this time, the actual performance optimization space of the cluster is still very large, and the cluster performance cannot be maximally utilized, and the business-side performance cannot be maximally improved.

[0062] For example, one day, the cluster experienced jitter. There were 10 business lines running in the cluster, and these business lines were all affected simultaneously. One minute after the jitter occurred, the cluster maintainer received a jitter warning and immediately launched a root cause investigation to locate and solve the problem. After 5 minutes, the cluster maintainer located the problem index and the cause of the problem. It took another 5 minutes to organize and optimize the suggestions and synchronize them to the business lines corresponding to the index to promote optimization. After receiving the optimization suggestions, the business side took 5 minutes to determine the optimization plan based on its own business situation. Finally, it took 30 minutes for the business side to actually implement the optimization plan, and the problem was solved. In this way, this cluster jitter affected a total of 10 business lines, and the jitter lasted for 46 minutes. Such a long and wide-ranging impact is completely unacceptable in business.

[0063] Based on this, before the cluster has a problem (cluster jitter), this application pre-optimizes the statement generation code and index structure through early warning inspections to improve the ability of cluster performance management. Thus, through prior optimization, it is possible to avoid discovering the need for optimization only after a cluster accident, and greatly reduce the probability of stability events occurring.

[0064] Among them, prior optimization refers to, before the cluster has a problem, by analyzing dimensions such as the index structure, data volume, and traffic, pre-discovering points that can be optimized and providing optimization solutions, so as to avoid performance bottlenecks caused by improper index design, excessive data volume, or abnormal traffic. Previously, it was often necessary to wait until a problem occurred online and affected a business before the problem could be discovered. And it took n days from discovery to the completion of optimization and going live. Adopting this solution can greatly avoid the occurrence of online failures and improve stability.

[0065] Figure 1 It is a schematic diagram of the functional module for optimizing cluster services provided by the embodiment of this application, as Figure 1 shown, including: a business layer, a data collection layer, an early warning agent (Agent) layer, and an early warning platform layer.

[0066] The business layer includes a business application (Application, APP), a gateway application (Gateway APP), and an ES cluster (Cluser). Among them, the business application routes query requests to the corresponding ES cluster through the corresponding gateway application, and reports the query requests (traffic data) to the data collection layer through the gateway application.

[0067] The data collection layer is used to collect query requests from the gateway application and collect cluster index information from the ES cluster.

[0068] The warning proxy layer is used to implement data analysis of various indicators and supports hot pluggability, allowing for the dynamic addition or deletion of analysis indicators. Analysis indicators can include, for example: shard health, high and low cluster compatibility, index data volume, fuzzy query, range query, deep paging, inefficient index structure, Queries Per Second (QPS), etc.

[0069] It should be noted that the design of the warning proxy with hot pluggability allows for the dynamic addition or removal of analysis indicators as needed, specifically addressing various factors that may lead to a decline in cluster performance, such as shard health, excessive index data volume, inefficient query patterns, etc., ensuring that the cluster always maintains a high-performance state.

[0070] The warning platform layer is used to collect the warning data generated by the warning proxy layer, associate the corresponding optimization solutions or cases according to the warning type, and send them to the corresponding maintenance party. After the maintenance party confirms the completion of the optimization, a warning regression verification is carried out.

[0071] In this embodiment, by integrating the four modules of the business layer, data collection layer, warning proxy layer, and warning platform layer, a closed-loop management system is formed. Thus, through gateway-side traffic collection, cluster index information collection, data cleaning and analysis, risk perception discovery, matching of risk item solutions, reaching the risk and solutions to the associated responsible person, the responsible person implementing the solution to the risk, and the platform verifying that the risk has been resolved, the business and cluster status are restored to health, forming a virtuous closed-loop. This not only improves the efficiency of problem identification but also ensures the effective implementation and effect tracking of optimization measures, systematically enhancing the ability of cluster performance management.

[0072] On the basis of Figure 1 the following, several specific embodiments are combined to illustrate the cluster service optimization method of the present application.

[0073] Figure 2 It is a flow diagram of the cluster service optimization method provided by the embodiment of the present application Figure 1 In this embodiment, the execution subject can be an electronic device, such as a computer, a tablet, etc.

[0074] As Figure 2 shown, the method may include:

[0075] S101. Obtain the request information of the query request for the target distributed service cluster and the cluster index information of the target distributed service cluster.

[0076] The target distributed service cluster can be an ES cluster, and the query request for the target distributed service cluster is used to request data query from the target distributed service cluster.

[0077] When a service application routes a query request to a target distributed service cluster through a corresponding gateway application, the gateway application can asynchronously record the request information of the query request. The request information includes: the request body domain-specific language (DSL) statement of the query request. The request body DSL statement is a statement generated by using a preset statement generation code. The cluster index information includes: the target index structure of the target distributed service cluster.

[0078] When querying data from a target distributed service cluster, a query request is usually converted into a request body domain-specific language (DSL) statement to query data from the target distributed service cluster based on the request body DSL statement.

[0079] Among them, the query request is also a read request. The target index structure of the target distributed service cluster refers to the storage structure of data in the target distributed service cluster. For example, it can include a digital index structure, a text (keyword) index structure, a nested index structure, etc.

[0080] In some embodiments, the target distributed service cluster can be any one of multiple distributed service clusters, or can be a service cluster selected from multiple distributed service clusters according to a preset rule. This embodiment does not make a special limitation on this.

[0081] S102. Perform statement warning analysis according to the request body DSL statement to generate first warning data.

[0082] Perform statement warning analysis on the request body DSL statement. If there is a request body DSL statement of the query request that does not meet the preset statement conditions, it is determined that the request body DSL statement is abnormal, and then first warning data is generated. The first warning data is used to indicate the abnormal type of the request body DSL statement of the query request.

[0083] S103. Perform structure warning analysis according to the target index structure and the request body DSL statement to generate second warning data.

[0084] Combined with the request body DSL statement, judge whether the target index structure is reasonable. If the request body DSL statement and the target index structure match, it means that the target index structure is reasonable. If the query type of the query request does not match the target index structure, it means that the target index structure is unreasonable, and then second warning data is generated. The second warning data is used to indicate that the target index structure does not match the request body DSL statement.

[0085] S104. Obtain the optimization information of the preset statement generation code and the optimization information of the target index structure respectively according to the first warning data and the second warning data, so as to optimize the preset statement generation code and the target index structure.

[0086] According to the first warning data, obtain the optimization information of the preset statement generation code. The optimization information of the preset statement generation code is used to optimize the preset statement generation code, so that for subsequent new query requests, the optimized statement generation code is adopted to generate the request body DSL statement of the new query request, and the request body DSL statement meets the preset statement conditions.

[0087] According to the second warning data, obtain the optimization information of the target index structure. The optimization information of the target index structure is used to optimize the target index structure, so that the optimized index structure matches the request body DSL statement of the new query request, that is, the optimized index structure is more reasonable.

[0088] In some embodiments, the generated optimization information of the preset statement generation code and the optimization information of the target index structure are pushed to the maintainer. The maintainer can view the optimization information of the preset statement generation code and the optimization information of the target index structure by clicking to view the details, and decide whether to optimize the preset statement generation code and the target index structure based on the optimization information of the preset statement generation code and the optimization information of the target index structure in combination with its own business.

[0089] It should be noted that the optimization information of the preset statement generation code can be an optimization plan or case of the preset statement generation code, and the optimization information of the target index structure can be an optimization plan or case of the target index structure. Since the business behaviors or index structures that cause cluster anomalies often follow certain rules, for example, the problems that have occurred are likely to occur in other business parties, and the points to be optimized and the optimization suggestions can be associated through fixed matching rules, providing optimization ideas and principles, which greatly reduces the difficulty of optimization. Therefore, multiple optimization information of the preset statement generation code and multiple optimization information of the target index structure can be pre-saved. According to the first warning data, determine the optimization information of the preset statement generation code that matches the first warning data from the multiple optimization information of the preset statement generation code, and according to the second warning data, determine the optimization information of the target index structure that matches the second warning data from the multiple optimization information of the target index structure.

[0090] In this embodiment, if the request body DSL statement does not meet the preset statement conditions or the target index structure is unreasonable, it may cause cluster jitter. Based on this, before the cluster has problems, this application optimizes the statement generation code and index structure in advance through early warning inspections to improve the ability of cluster performance management. In addition, based on real traffic data (query requests), precise analysis is carried out to ensure high accuracy and low misjudgment rate of the analysis results, so that the proposed optimization measures are more accurate and effective, directly improving the cluster processing ability.

[0091] Figure 3 Flow diagram of the cluster service optimization method provided by the embodiment of this application Figure 2 , such as Figure 2 shown. In an optional embodiment, the above step S102, performing statement early warning analysis according to the request body DSL statement to generate first early warning data, may include:

[0092] S201. Determine the target query type and paging depth of the query request according to the request body DSL statement.

[0093] It should be noted that for the target distributed service cluster, the paging storage mechanism is a mechanism for processing a large amount of data in the cluster. The data in the cluster is divided into multiple shards, and each shard stores data independently. When performing a paging query, each shard independently queries based on the starting offset (from) and the size of each page, and collects the query structures of each shard and combines them into a global result set, and then intercepts the data of the size of each page starting from the starting offset from the global structure set as the final return data.

[0094] Determine the target query type, starting offset, and size of each page of the query request from the request body DSL statement. Among them, the target query type can be, for example, an exact query (TermQuery), an equivalent query, a fuzzy query, or a range query. Among them, an exact query is used to perform an exact match query on a certain field in the cluster to ensure that the query result exactly matches the specified value. An equivalent query is used to find documents that exactly match the specified value. An equivalent query is usually used in scenarios that require exact matching, such as finding specific numerical values, boolean values, or exact text values. A fuzzy query is used to find documents similar to the specified value in the cluster, and is suitable for scenarios where spelling mistakes or variant words are processed. A range query is used to find documents whose field values are within a specified range, and is suitable for types such as numerical values and dates.

[0095] The paging depth is equal to the sum of the starting offset and the size of each page. For example, if from is 0 and size is 10, then the paging depth is 10.

[0096] It should be noted that for the same query request, the request body DSL statement can be written in different ways, that is, a query request can correspond to multiple different request body DSL statements, which is specifically determined by the preset code generation statement. This embodiment does not make special limitations on this.

[0097] S202. If the target query type is a fuzzy query type and / or the paging depth exceeds a preset depth threshold, generate first warning data.

[0098] If the target query type is a fuzzy query type, such as Wildcard, and / or the paging depth exceeds a preset depth threshold, generate first warning data. Among them, the content of the query request corresponding to the fuzzy query type is usually relatively long. Such requests will cause the overall CPU of the cluster to occupy 100%, seriously affecting stability. If the paging depth exceeds the preset depth threshold, it means that the sharding mechanism of the target distributed service cluster will amplify the impact brought by deep paging, occupy a large amount of memory, and may directly return an error affecting the cluster service. Among them, the preset depth threshold can be, for example, 10,000.

[0099] Among them, the first warning data is used to indicate that the target query type is a fuzzy query type and / or the paging depth exceeds a preset depth threshold. The optimization information of the preset statement generation code is used to optimize the preset statement generation code, so as to generate a new query request body DSL statement by using the optimized statement generation code. The query type in this request body DSL statement is a non-fuzzy query type and the paging depth does not exceed the preset depth threshold.

[0100] In this embodiment, before the cluster has problems, by inspecting the request body DSL statement and optimizing the preset statement generation code, it is possible to optimize the statement generation code in advance through the warning inspection method, and improve the ability of cluster performance management.

[0101] Figure 4 The flow diagram of the cluster service optimization method provided by the embodiment of the present application Figure 3 , such as Figure 4 shown. In an alternative embodiment, the above step S103, performing structural warning analysis according to the target index structure and the request body DSL statement to generate second warning data, may include: step S301 and / or step S302-S303:

[0102] S301. If the target index structure is an object array index structure or a join index structure, generate second warning data.

[0103] Among them, the object array index structure is like nested, and the join index structure is like join type. If the target index structure is an object array index structure or a join index structure, second warning data is generated. Among them, the object array index structure and the join index structure are inefficient index structures, that is, unreasonable index structures, which have a significant impact on index reading and writing. Such structures will bring hundreds of times of resource overhead. Therefore, they should be avoided as much as possible.

[0104] Taking the nested structure as an example, the performance of the nested structure is hundreds of times weaker than that of the keyword structure, and it will bring extremely serious data expansion problems. Therefore, it is necessary to process and optimize unreasonable index structures in a timely manner.

[0105] Among them, the second warning data is used to indicate that the target index structure is an object array index structure or a join index structure, and the optimization information of the target index structure is used to optimize the target index structure. For example, optimizing the nested structure into a keyword structure.

[0106] S302. According to the preset matching relationship between the query type and the index structure, determine whether the target query type and the target index structure of the query request match.

[0107] S303. If the target query type and the target index structure do not match, generate second warning data.

[0108] Among them, the query type and the index structure have a preset matching relationship. Determine the target query type of the query request from the request body DSL statement. According to the target query type of the query request, query this matching relationship, and judge the index structure matched by the target query type of the query request. If the index structure matched by the target query type is the target index structure, it is determined that the target query type and the target index structure match. If the index structure matched by the target query type is not the target index structure, it is determined that the target query type and the target index structure do not match.

[0109] It should be noted that in the case where the query type and the index structure match, the query performance is better, and in the case where the query type and the index structure do not match, the query performance is worse.

[0110] If the target query type and the target index structure do not match, generate second warning data. The second warning data is used to indicate that the target query type and the target index structure do not match, and the optimization information of the target index structure is used to optimize the target index structure so that the optimized index structure matches the query type of the query request and improves the query performance.

[0111] Among them, the target index structure refers to the storage structure of data in the target distributed service cluster, which may include, for example, a digital index structure, a text (keyword) index structure, a nested index structure, etc. For example, the query type of a range query matches the digital index structure, and the query performance is better, but the query type of an exact query does not match the digital index structure, and the query performance is poor.

[0112] In this embodiment, before the cluster has a problem, by using the inspection request body DSL statement and the target index structure and optimizing the target index structure, it is possible to pre-optimize the target index structure in the way of warning inspection, and improve the ability of cluster performance management.

[0113] Figure 5 It is a flowchart of the cluster service optimization method provided by the embodiment of the present application Figure 4 , as Figure 5 shown, in an optional implementation manner, the cluster index information further includes: the number of nodes, the number of shards, and the index data of the target distributed service cluster.

[0114] Among them, the number of nodes of the target distributed service cluster is obtained, the number of shards is the total number of multiple shards of the target distributed service cluster, each shard is used to store data independently, the shards are deployed on nodes, and the index data includes the total number of data and the total size of data in the target distributed service cluster, and the total size of data refers to the memory occupied by the data.

[0115] The method may further include:

[0116] S401. Perform shard health warning analysis according to the number of nodes, the number of shards, and the index data to obtain the third warning data.

[0117] Check whether the number of nodes and the number of shards are consistent. If they are not consistent, it means that there may be a problem of uneven hot index shards, which may cause some nodes to be overloaded and affect the overall cluster performance. For example, the number of shards is 2 and the number of nodes is 3. In this way, 2 shards are deployed on 2 of the nodes, and no shard is deployed on the other node, resulting in uneven node load.

[0118] According to the number of shards and the total size of data, calculate the size of a single shard. The size of a single shard refers to the memory occupied by the data stored in a single shard. The size of a single shard is equal to the total size of data / the number of shards. The size of a single shard can be maintained within a preset size range, such as between 30G and 50G. If the size of a single shard is too large, the cluster's fault recovery speed may become slow. If the size of a single shard is too small, there may be a very large number of shards, because each shard will occupy some CPU and memory, resulting in problems such as read / write performance and memory shortage.

[0119] Calculate the single - shard data volume based on the number of shards and the total data volume. The single - shard size refers to the amount of data stored in a single shard. The single - shard data volume is equal to the total data volume divided by the number of shards. If the single - shard data volume exceeds the first data volume threshold, it will greatly increase the query time, especially in scenarios where the inverted index chain is too long, and the merging of the inverted index chain will incur huge overhead.

[0120] It should be noted that a suitable number of shards needs to be given in advance. At the same time, in order to ensure the balance of data sharding, the number of nodes also needs to be taken into account when determining the number of shards.

[0121] By checking whether the number of nodes is consistent with the allocation number, whether the single - shard size is within the preset size range, and whether the single - shard data volume is within the preset data volume range, sharding health warnings are issued to generate the third warning data. If at least one of the conditions that the number of nodes is inconsistent with the allocation number, the single - shard size is not within the preset size range, and the single - shard data volume exceeds the first data volume threshold is met, the third warning data is generated. The third warning data is used for the situation where the number of nodes is inconsistent with the allocation number, the single - shard size is not within the preset size range, and the single - shard data volume exceeds the first data volume threshold.

[0122] S402. Conduct data volume warning analysis based on the index data to generate the fourth warning data.

[0123] The index data includes the total data volume and the total data size of the data in the target distributed service cluster. If the total data volume exceeds the second data volume threshold and / or the total data size exceeds the preset size threshold, the fourth warning data is generated. The fourth warning data is used to indicate that the total data volume exceeds the second data volume threshold and / or the total data size exceeds the preset size threshold.

[0124] S403. Obtain the optimization information of the number of shards and the optimization information of the index data according to the third warning data and the fourth warning data respectively, so as to optimize the number of shards and the index data.

[0125] According to the third warning data, obtain the optimization information of the number of shards. The optimization information of the number of shards is used to optimize the number of shards. According to the fourth warning data, obtain the optimization information of the index data. The optimization information of the index data is used to optimize the index data. For example, by means of splitting the index data, reducing unnecessary redundant fields, etc., the total data volume and the total data size are reduced.

[0126] It should be noted that the optimization information of the number of shards can be an optimization plan or case for the number of shards, and the optimization information of the index data can be an optimization plan or case for the index data. According to the third warning data, query the optimization information of the number of shards that matches the third warning data, and according to the fourth warning data, query the optimization information of the index data that matches the fourth warning data.

[0127] In this embodiment, by optimizing the number of nodes and the number of shards during patrol inspection, the utilization rate of resources and performance can be maximized. For example, before optimization, if the average load of 5 nodes in the cluster is 80%. After optimization, the average load is 50%. At this time, 2 cluster nodes can be reduced to save hardware costs; for another example, before optimization, the average load of 5 nodes is 80%. After optimization, the average load is 50%. The resource occupancy rate is greatly reduced, and the cluster stability is greatly improved; for another example, before optimization, the resource occupancy rates of 3 nodes are 80%, 80%, and 50% respectively. After optimization, the resource occupancy rates of 3 nodes are 65%, 65%, and 65% respectively. Following the principle of the shortest board in a wooden barrel, the resource occupancy rate is increased by 15%.

[0128] Figure 6 Flow schematic of the cluster service optimization method provided by the embodiment of the present application Figure 5 , such as Figure 6 shown, in an alternative embodiment, the method may further include:

[0129] S501. According to the target index structure and the request body DSL statement, obtain the underlying query request of the request body DSL statement under the target distributed service cluster.

[0130] When querying data from the target distributed service cluster, the request body DSL statement will be converted into an underlying query request to optimize the query performance. Different target index structures and query types are converted into different underlying query requests. Therefore, the target query type can be determined from the request body DSL statement, and according to the target index structure and the target query type, query the corresponding relationship between the preset index structure, query type, and underlying query request, and obtain the underlying query request of the request body DSL statement under the target distributed service cluster.

[0131] For example, if the index structure is a text index structure and the query type is an exact query, then the underlying query request of the request body DSL statement under the target distributed service cluster is a range query request.

[0132] S502. If the underlying query request is a range query request, generate the fifth warning data.

[0133] If the underlying query request is a range query request (PointRangeQuery), generate the fifth warning data, and the fifth warning data is used to indicate that the underlying query request is a range query request. Among them, the range query request is an optimization method for numerical range queries and is usually used to process exact matches and range queries of numerical fields.

[0134] S503. According to the fifth warning data, obtain the optimization information of the preset statement generation code and / or the optimization information of the target index structure.

[0135] According to the fifth warning data, obtain the optimization information of the preset statement generation code to optimize the preset statement generation code, so as to generate the request body DSL statement of the new query request by using the optimized statement generation code, and based on the target index structure and the request body DSL statement of the new query request, obtain the underlying query request of the request body DSL statement of the new query request in the target distributed service cluster, and this underlying query request is not a range query request. For example, the performance of the PointRangeQuery request using the BKD-Tree query is hundreds of times lower than that of the inverted index. Such requests should be optimized to use the TermQuery to perform the inverted index query, that is, execute the TermQuery query through the inverted index to quickly locate the documents containing specific terms.

[0136] And / or, according to the fifth warning data, obtain the optimization information of the target index structure to optimize the target index structure, so that based on the optimized index structure and the request body DSL statement of the new query request, obtain the underlying query request of the request body DSL statement of the new query request in the target distributed service cluster, and this underlying query request is not a range query request.

[0137] In some embodiments, according to the first warning data and the fifth warning data, obtain the optimization information of the preset statement generation code, and / or according to the second warning data and the fifth warning data, obtain the optimization information of the preset statement generation code. Among them, the optimization information of the preset statement generation code that matches the first warning data and the fifth warning data can be determined from various optimization information of the preset statement generation code according to the first warning data and the fifth warning data, and the optimization information of the target index structure that matches the second warning data and the fifth warning data can be determined from various optimization information of the target index structure according to the second warning data and the fifth warning data.

[0138] In an alternative embodiment, the method may further include:

[0139] S504. If the underlying query request is a minimum match query request and the target distributed service cluster is the updated service cluster, then generate the sixth warning data.

[0140] Among them, the minimum match query request is such as minimum_should_match. If the underlying query request is a minimum match query request and the target distributed service cluster is the updated service cluster, that is, the target distributed service cluster is the upgraded or switched service cluster, then generate the sixth warning data, and the sixth warning data is used to indicate that the underlying query request is a minimum match query request and the target distributed service cluster is the updated service cluster.

[0141] Generally, for the minimum matching query request, there is a problem of inconsistent default values in the high and low cluster versions. Such requests will cause inconsistent retrieval results when the cluster is upgraded or switched. For example, only 10 data items can be retrieved in the low-version cluster, but 12 data items can be retrieved for the same request in the high-version cluster.

[0142] S505. Obtain the optimization information of the preset statement generation code and / or the optimization information of the target index structure according to the sixth warning data.

[0143] According to the sixth warning data, obtain the optimization information of the preset statement generation code to optimize the preset statement generation code, so that the optimized statement generation code is used to generate the request body DSL statement of the new query request, and based on the target index structure and the request body DSL statement of the new query request, obtain the underlying query request of the request body DSL statement of the new query request under the target distributed service cluster, and this underlying query request is not a minimum matching query request.

[0144] And / or, according to the sixth warning data, obtain the optimization information of the target index structure to optimize the target index structure, so that based on the optimized index structure and the request body DSL statement of the new query request, obtain the underlying query request of the request body DSL statement of the new query request under the target distributed service cluster, and this underlying query request is not a minimum matching query request.

[0145] In some embodiments, obtain the optimization information of the preset statement generation code according to the first warning data and the sixth warning data, and / or obtain the optimization information of the target index structure according to the second warning data and the sixth warning data. Among them, the optimization information of the preset statement generation code that matches the first warning data and the sixth warning data can be determined from various optimization information of the preset statement generation code according to the first warning data and the sixth warning data, and the optimization information of the target index structure that matches the second warning data and the sixth warning data can be determined from various optimization information of the target index structure according to the second warning data and the sixth warning data.

[0146] In this embodiment, before the cluster has problems, by inspecting the underlying query request and optimizing the preset statement generation code and / or the target index structure, it is possible to pre-optimize the statement generation code and / or the target index structure in the form of warning inspection, and improve the ability of cluster performance management.

[0147] Figure 7 It is a flowchart of the cluster service optimization method provided by the embodiment of the present application Figure 6 , such as Figure 7 shown. In an alternative embodiment, the request information further includes: a request index corresponding to the query request, and the request index is used to indicate the target index resource in the target distributed service cluster.

[0148] Among them, the target index resource in the target distributed service cluster refers to the data resource in the target distributed service cluster, which can be, for example, an index data table. The request index may include: the table name (index name) of the index data table. That is, data query is performed from the index data table in the target distributed service cluster based on the query request.

[0149] The method may further include:

[0150] S601. Calculate the query rate for the target index resource according to the request index.

[0151] S602. If the query rate exceeds the preset query rate threshold, generate the seventh warning data.

[0152] Calculate the query rate for the target index resource according to the request index. This query rate can be the query per second rate QPS. That is, calculate the query rate based on multiple query requests. If this query rate exceeds the preset query rate threshold, generate the seventh warning data, where the seventh warning data is used to indicate that the query rate for the target index resource exceeds the preset query rate threshold.

[0153] It should be noted that if the query rate exceeds the preset query rate threshold, it indicates that there may be abnormal traffic (such as crawlers, BUGs). In this case, once it occurs, early warning can be given through the judgment of the query rate.

[0154] In an optional implementation manner, in step S601 above, calculating the query rate for the index resource according to the request index may include:

[0155] According to the target query type of the query request, replace the values in the request body DSL statement with the placeholders corresponding to the target query type to obtain the target request body DSL signature;

[0156] Calculate the query rate under the target request body DSL signature according to the request index.

[0157] Determine the target query type of the query request from the request body DSL statement, and according to the target query type, replace the values in the request body DLS statement with the placeholders corresponding to the target query type to obtain the target request body DSL signature, where there is a corresponding relationship between the query type and the placeholder, and the placeholder refers to a special symbol or identifier used in programming, configuration files, templates, or query statements to represent a value or position to be filled.

[0158] For example, a request body DSL statement is expressed as:

[0159]

[0160] Another request body DSL statement is expressed as:

[0161]

[0162]

[0163] If the target query types corresponding to the above two request body DSL statements for the query request are both exact queries, then replace the values in the above two request body DSL statements with the placeholder (Value) to obtain the target request body DSL signature, which is expressed as:

[0164]

[0165] It can be seen from this that the request body DSL signature is obtained through DSL cleaning. For different request body DSL statements of the same target query type, by erasing the values and replacing them with placeholders, it is used to represent the same type of DSL query statements with only different values but the same query type. Among them, the request body DSL statements with the same request DSL signature often have very similar commonalities.

[0166] Then, according to the request index, calculate the query rate for the target index resource under the target request body DSL signature. This query rate refers to the query rate for the target index resource when using the target request body DSL signature to perform data query on the target index resource. That is to say, based on multiple query requests with the same query type, and the request index of each query request is the target index resource, then after cleaning the request body DSL statements of each query request, the same request body DSL signature is obtained, and the query rate for the target index resource under the same request body DSL signature is calculated.

[0167] S603. According to the seventh warning data, obtain the optimization information of the target index resource to optimize the access to the target index resource.

[0168] According to the seventh warning data, obtain the optimization information of the target index resource. This optimization information is used to optimize the access to the target index resource. That is to say, when the query rate exceeds the preset query rate threshold, by optimizing the access to the target index resource, abnormal traffic is avoided.

[0169] It should be noted that if it is further determined that the query request is a normal access, the query traffic for the target index resource can be restricted. If it is determined that the query request is an external access, the query request for the target index resource can be intercepted. If it is determined that it is a system BUG, the system BUG can be repaired to optimize the access to the target index resource.

[0170] In some embodiments, the optimization information of the target index resource may be an optimization solution or case of the target index resource. According to the seventh warning data, the optimization information of the target index resource that matches the seventh warning data is determined from various optimization information of the target index resource.

[0171] In this embodiment, before a problem occurs in the cluster, by means of the query rate, the underlying query requests for inspection are optimized and the access to the target index resource is optimized, so that the access to the target index resource can be optimized in advance through the warning inspection, and the ability of cluster performance management can be improved.

[0172] Figure 8 Schematic flow of the cluster service optimization method provided by the embodiments of the present application Figure 7 , such as Figure 8 shown, in an alternative embodiment, the request information further includes: a request link identifier of the query request.

[0173] Among them, the request link identifier (requestId) of the query request is used to indicate the request link corresponding to the query request.

[0174] The method may further include:

[0175] S701. Obtain the service application corresponding to the query request according to the request link identifier of the query request.

[0176] S702. Push the optimization information of the preset statement generation code and the optimization information of the target index structure to the application maintainer of the service application corresponding to the query request.

[0177] Among them, there is a corresponding relationship between the request link and the access application. According to the request link identifier of the query request, this corresponding relationship can be queried to determine the service application corresponding to the query request. The access application is the application that triggers the query request for the target distributed service cluster.

[0178] Then, push the optimization information of the preset statement generation code and the optimization information of the target index structure to the application maintainer of the service application corresponding to the query request, so that the application maintainer can know the optimization solutions for the preset statement generation code and the target index structure, and push for corresponding optimization.

[0179] In some embodiments, the optimization information of the shard quantity, the optimization information of the index data, and the optimization information of the target index resource may also be pushed to the application maintainer, so that the application maintainer can know the optimization solutions for the shard quantity, the index data, and the target index resource, and push for corresponding optimization. Additionally, after the optimization is completed, a regression inspection of the optimization result can be performed. Among them, the optimization of the cluster index information and the target index resource can be executed by the cluster maintainer, and the optimization of the preset statement generation code can be executed by the application maintainer.

[0180] In this embodiment, problems existing in the cluster and the business usage mode are discovered a priori through patrol inspection and early warning. When problems are discovered through early warning, optimization suggestions are automatically matched and pushed to personnel, reducing the optimization cost. Moreover, the early warning information directly discovers and pushes to the responsible person (application maintenance party), eliminating the need for the cluster maintainer to analyze one by one and prompt the responsible person to optimize, greatly reducing the pressure on the cluster maintainer.

[0181] In the past, after the cluster maintainer discovered a problem, it took time to find the accessed application and the application maintenance party based on the cluster traffic and logs, time to communicate with the responsible person about the problem location and principle, and time to communicate with the application maintenance party about the solution. However, with this solution, there is no need for the cluster maintainer to manually notify, saving time and realizing automatic early warning and optimization suggestion push. It changes the status quo of the inefficient and lagging one-by-one promotion of application maintenance by the cluster maintainer in the past, and is completely insensitive to the business usage of the business side. Business traffic information is collected on the gateway application side, and the business side can access the risk analysis function without any modification.

[0182] It should be noted that the request information may further include: the application name of the business application for the query request, the request source IP, the information of the distributed business cluster to which the request is actually routed by the gateway application, the request processing time, the request start and end times, the request result status, the request method (method), the request index type (type), and the request document ID.

[0183] Among them, the request source IP is the IP address of the device sending the query request, the request processing time is the time between receiving the query request and returning a response to the business application, the request start and end times include the time of receiving the query request and the time of returning the response, the request result status is used to indicate whether the query request successfully returns a response, including success and failure, and the request method is used to indicate the query operation performed on the distributed business cluster.

[0184] Therefore, the target distributed business cluster is determined from multiple distributed business clusters based on the request result status and the request processing time. The request result status corresponding to the target distributed business cluster is failure and the request processing time is relatively long. That is to say, a distributed business cluster is selected for optimization according to the priority.

[0185] The request index type is a subset of the request index and is used to indicate where to query data from the index resources (index data tables) indicated by the request index. The request document includes the above request information. By saving the above request information, it can be called when subsequent services are needed.

[0186] Figure 9 It is a schematic structural diagram of the cluster service optimization device provided by the embodiment of the present application, and this device can be integrated in an electronic device.

[0187] As Figure 9 shown, the device may include:

[0188] An acquisition module 801, configured to acquire a query request for a target distributed service cluster and cluster index information of the target distributed service cluster, where the cluster index information includes: a target index structure of the target distributed service cluster;

[0189] A processing module 802, configured to generate code by using a preset request body domain-specific language (DSL) statement, and generate request information of the query request, where the request information includes: the request body domain-specific language (DSL) statement of the query request;

[0190] The processing module 802 is further configured to perform statement warning analysis according to the request body DSL statement, and generate first warning data;

[0191] The processing module 802 is further configured to perform structure warning analysis according to the target index structure and the request body DSL statement, and generate second warning data;

[0192] The acquisition module 801 is configured to respectively acquire optimization information of the preset statement generation code and optimization information of the target index structure according to the first warning data and the second warning data, so as to optimize the preset statement generation code and the target index structure.

[0193] In an optional embodiment, the processing module 802 is specifically configured to:

[0194] Determine a target query type and a paging depth of the query request according to the request body DSL statement;

[0195] If the target query type is a fuzzy query type, and / or, the paging depth exceeds a preset depth threshold, then generate first warning data.

[0196] In an optional embodiment, the processing module 802 is specifically configured to:

[0197] If the target index structure is an object array index structure or a join index structure, then generate second warning data; and / or,

[0198] Judge whether the target query type and the target index structure of the query request match according to a preset matching relationship between the query type and the index structure;

[0199] If the target query type and the target index structure do not match, then generate second warning data.

[0200] In an optional embodiment, the cluster index information further includes: the number of nodes, the number of shards, and index data of the target distributed service cluster; the processing module 802 is further configured to:

[0201] Perform shard health warning analysis based on the number of nodes, the number of shards, and index data to obtain third warning data;

[0202] Perform data volume warning analysis based on index data to generate fourth warning data;

[0203] Based on the third warning data and the fourth warning data, obtain optimization information for the number of shards and optimization information for index data respectively to optimize the number of shards and index data.

[0204] In an optional implementation manner, the processing module 802 is further configured to:

[0205] Obtain the underlying query request of the request body DSL statement under the target distributed service cluster according to the target index structure and the request body DSL statement;

[0206] If the underlying query request is a range query request, generate fifth warning data;

[0207] According to the fifth warning data, obtain optimization information for the preset statement generation code and / or optimization information for the target index structure.

[0208] In an optional implementation manner, the processing module 802 is further configured to:

[0209] If the underlying query request is a minimum matching query request and the target distributed service cluster is the updated service cluster, generate sixth warning data;

[0210] According to the sixth warning data, obtain optimization information for the preset statement generation code and / or optimization information for the target index structure.

[0211] In an optional implementation manner, the request information further includes: a request index corresponding to the query request, and the request index is used to indicate the target index resource in the target distributed service cluster. The processing module is further configured to:

[0212] Calculate the query rate for the target index resource according to the request index;

[0213] If the query rate exceeds the preset query rate threshold, generate seventh warning data;

[0214] According to the seventh warning data, obtain optimization information for the target index resource to optimize the access to the target index resource.

[0215] In an optional implementation manner, the processing module 802 is further configured to:

[0216] According to the target query type of the query request, replace the values in the request body DSL statement with placeholders corresponding to the target query type to obtain a target request body DSL signature;

[0217] Calculate the query rate under the DSL signature of the target request body according to the request index.

[0218] In an optional embodiment, the request information further includes: the request link identifier of the query request; the processing module 802 is further configured to:

[0219] Obtain the service application corresponding to the query request according to the request link identifier of the query request.

[0220] Push the optimization information of the preset statement generation code and the optimization information of the target index structure to the application maintainer of the service application corresponding to the query request.

[0221] For the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiments, which will not be elaborated here.

[0222] Figure 10 It is a schematic structural diagram of the electronic device provided by the embodiment of the present application. As Figure 10 shown, the device may include: a processor 901, a memory 902, and a bus 903. The memory 902 stores machine-readable instructions executable by the processor 901. When the electronic device runs, the processor 901 communicates with the memory 902 through the bus 903, and the processor 901 executes the machine-readable instructions to execute the above method.

[0223] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the above method.

[0224] In the embodiment of the present application, when the computer program is run by a processor, it may also execute other machine-readable instructions to execute the other methods described in the embodiments. For the specific method steps and principles executed, refer to the descriptions in the embodiments, which will not be elaborated in detail here.

[0225] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces. The indirect coupling or communication connection of the device or unit may be in an electrical, mechanical or other form.

[0226] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0227] In addition, each functional unit in the embodiments provided in this application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0228] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0229] It should be noted that: similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0230] Finally, it should be noted that: the above-described embodiments are only specific implementation manners of this application, used to illustrate the technical solutions of this application, rather than limiting it. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed in this application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for optimizing cluster services, characterized in that, Including: Obtain the request information of the query request for the target distributed service cluster and the cluster index information of the target distributed service cluster. The request information includes: the request body domain-specific language (DSL) statement of the query request, and the request body DSL statement is a statement generated by using a preset statement generation code. The cluster index information includes: the target index structure of the target distributed service cluster; Perform statement warning analysis based on the request body DSL statement to generate first warning data; Perform structure warning analysis based on the target index structure and the request body DSL statement to generate second warning data; According to the first warning data and the second warning data, obtain the optimization information of the preset statement generation code and the optimization information of the target index structure respectively, so as to optimize the preset statement generation code and the target index structure.

2. The method according to claim 1, characterized in that The performing statement warning analysis based on the request body DSL statement to generate first warning data includes: Determine the target query type and paging depth of the query request according to the request body DSL statement; If the target query type is a fuzzy query type, and / or the paging depth exceeds a preset depth threshold, then generate the first warning data.

3. The method according to claim 1, wherein The performing structure warning analysis based on the target index structure and the request body DSL statement to generate second warning data includes: If the target index structure is an object array index structure or a join index structure, then generate the second warning data; and / or Judge whether the target query type of the query request and the target index structure match according to the preset matching relationship between the query type and the index structure; If the target query type and the target index structure do not match, then generate the second warning data.

4. The method according to claim 1, wherein The cluster index information further includes: the number of nodes, the number of shards, and the index data of the target distributed service cluster; The method further includes: Perform shard health warning analysis based on the number of nodes, the number of shards, and the index data to obtain third warning data; Perform data volume warning analysis based on the index data to generate fourth warning data; According to the third warning data and the fourth warning data, obtain the optimization information of the number of shards and the optimization information of the index data respectively, so as to optimize the number of shards and the index data.

5. The method according to claim 1, wherein The method further includes: Obtain the underlying query request of the request body DSL statement in the target distributed service cluster according to the target index structure and the request body DSL statement; If the underlying query request is a range query request, then generate fifth warning data; According to the fifth warning data, obtain the optimization information of the preset statement generation code and / or the optimization information of the target index structure.

6. The method according to claim 5, characterized in that, The method further includes: If the underlying query request is a minimum match query request, and the target distributed service cluster is an updated service cluster, then generate sixth warning data; According to the sixth warning data, obtain the optimization information of the preset statement generation code and / or the optimization information of the target index structure.

7. The method according to claim 1, characterized in that The request information further includes: a request index corresponding to the query request, where the request index is used to indicate a target index resource in the target distributed service cluster, and the method further includes: Calculating a query rate for the target index resource according to the request index; If the query rate exceeds a preset query rate threshold, generating seventh warning data; Obtaining optimization information of the target index resource according to the seventh warning data to optimize access to the target index resource.

8. The method according to claim 7, characterized in that The calculating a query rate for the target index resource according to the request index includes: Replacing values in the DSL statement of the request body with placeholders corresponding to the target query type according to the target query type of the query request to obtain a target request body DSL signature; Calculating the query rate under the target request body DSL signature according to the request index.

9. The method according to claim 1, characterized in that, The request information further includes: a request link identifier of the query request; and the method further includes: Obtaining a service application corresponding to the query request according to the request link identifier of the query request; Pushing the optimization information of the preset statement generation code and the optimization information of the target index structure to the application maintainer of the service application corresponding to the query request.

10. A cluster service optimization device, characterized in that, including: An obtaining module, configured to obtain a query request for a target distributed service cluster and cluster index information of the target distributed service cluster, where the cluster index information includes: a target index structure of the target distributed service cluster; A processing module, configured to generate request information of the query request by using a preset request body domain-specific language (DSL) statement generation code, where the request information includes: the DSL statement of the request body of the query request; The processing module is further configured to perform statement warning analysis according to the DSL statement of the request body to generate first warning data; The processing module is further configured to perform structure warning analysis according to the target index structure and the DSL statement of the request body to generate second warning data; An obtaining module, configured to obtain the optimization information of the preset statement generation code and the optimization information of the target index structure respectively according to the first warning data and the second warning data to optimize the preset statement generation code and the target index structure.

Images