Containerized deployment method, tool and system for graph database

Abstract

The invention discloses a containerized deployment method, tool and system of a graph database, the method is used for carrying out self-adaptive deployment at a current node based on key parameters input by a user, and the method comprises the following steps: carrying out matching with preset dynamic templates based on the key parameters to obtain corresponding candidate templates, each dynamic template comprises a placeholder, and the placeholder is matched with the current node; each placeholder corresponds to an assignment rule and a dependency relationship; taking the corresponding placeholders in the candidate template as points and the dependency relationship as edges to generate a corresponding dependency graph which is a directed acyclic graph; based on the dependency graph, deducing actual values corresponding to the placeholders in sequence according to corresponding assignment rules, and generating corresponding deployment parameters; and carrying out container deployment on the current node based on the deployment parameters. Compared with an existing static template based on manual adjustment or simple default value coverage, the method has the advantages that the required parameters are automatically deduced through the design of the dynamic template and the dependency graph, and the deployment efficiency is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and in particular, to a method, tool and system for containerized deployment of a graph database. Background Art

[0002] Traditional graph database deployment tools, such as installation scripts, deployment control panels, etc., all rely on manual filling of a large number of configuration items, and in different production environments, manual adjustment of the configuration according to the actual situation is required, which is difficult to quickly adapt in complex environments, and there are problems of complex deployment, error-prone, and difficult to maintain. Summary of the Invention

[0003] In view of the disadvantages of the prior art that a large number of configuration items need to be filled in manually for the deployment of a graph database, with low efficiency, error-prone, and difficult to maintain, the present invention provides a method, tool and system for containerized deployment of a graph database.

[0004] To solve the above technical problems, the present invention is solved by the following technical solutions: In a first aspect, the present application proposes a method for containerized deployment of a graph database, which performs adaptive deployment on the current node based on key parameters input by a user, and includes the following steps: Based on the key parameters, match with a preset dynamic template to obtain a corresponding candidate template, each dynamic template includes placeholders, and the assignment rules and dependency relationships corresponding to each placeholder; Use the placeholders corresponding to the obtained candidate template as nodes, and the dependency relationships as edges to generate a corresponding dependency graph, and the dependency graph is a directed acyclic graph; Based on the dependency graph, sequentially deduce the actual values corresponding to each placeholder according to the corresponding assignment rules, and generate corresponding deployment parameters; Perform container deployment on the current node based on the deployment parameters; The dependency graph includes placeholders corresponding to adaptive memory configuration, and the step of assigning values to the placeholders corresponding to adaptive memory configuration based on the corresponding assignment rules is: Perform load simulation on the memory of the current node to obtain corresponding memory metrics; Predict the memory requirements based on the memory metrics to obtain corresponding prediction results; Generate an adaptive memory configuration corresponding to the current node based on the prediction results.

[0005] Through the design of dynamic templates and dependency graphs, the user only needs to configure a small number of key parameters, and other configuration parameters can be automatically deduced based on the matching dynamic templates, reducing manual intervention and improving deployment efficiency; Compared with the existing static templates based on manual adjustment or simple default value overwriting, the dynamic templates provided in this application embed corresponding assignment rules. For some parameters corresponding to memory requirements, it can automatically predict memory requirements based on the assignment rules and perform adaptive configuration based on the prediction results.

[0006] As an implementable manner, when performing multi-node deployment, it further includes the step of selecting NUMA nodes for container deployment, specifically: Each NUMA node performs performance evaluation based on a preset performance evaluation rule to obtain a corresponding performance score; Based on the performance scores corresponding to each NUMA node and a preset deployment decision rule, several NUMA nodes are selected as target nodes; The central node distributes the key parameters input by the user to each target node, and each target node performs adaptive deployment based on the key parameters.

[0007] In the multi-node deployment scenario, different from the traditional container deployment that manually specifies NUMA binding through -cpu, this application performs performance evaluation on each NUMA node, selects the NUMA node with the best performance for binding, maximizes the local memory access efficiency, and reduces the cross-node communication latency.

[0008] Note: NUMA, Non-Uniform Memory Access, non-uniform memory access.

[0009] As an implementable manner, when performing multi-node deployment, the central node distributes the key parameters input by the user to each NUMA node, and each NUMA node automatically makes a deployment decision based on the key parameters, and cancels the deployment or performs adaptive deployment based on the obtained deployment decision; Among them, a dependency graph corresponding to the current node is generated based on the key parameters. The dependency graph includes placeholders corresponding to cpu_bindings, and the assignment rules corresponding to the placeholders include performance evaluation rules and deployment decision rules; When assigning values to the placeholders corresponding to cpu_bindings based on the corresponding assignment rules: Perform performance evaluation on the current node based on the performance evaluation rule to obtain the performance score corresponding to the current node; Generate a deployment decision corresponding to the current node based on the deployment decision rule and the performance scores corresponding to each NUMA node; When the deployment decision is to cancel the deployment, end the derivation of the dependency graph; When the deployment decision is to perform container deployment, assign values to the placeholders corresponding to cpu_bindings based on the set of CPU logical cores of the current node, and continue to deduce the actual values corresponding to the subsequent placeholders based on the corresponding dependency graph.

[0010] Similarly, in the multi-node deployment scenario, the present application evaluates the performance of each NUMA node and selects the NUMA node with the best performance for binding. This solution uses the performance evaluation rule and the deployment decision rule as the assignment rules corresponding to the cpu_bindings placeholder. Therefore, when deducing the cpu_bindings placeholder, it intelligently decides whether to perform container deployment on the current node based on the corresponding assignment rules.

[0011] Note: cpu_bindings is a parameter used to bind a process or thread to a specific CPU core.

[0012] As an implementable manner, the steps for the current node to perform performance evaluation based on the performance evaluation rule include: Perform a load simulation on the CPU of the current node to obtain the performance metrics corresponding to the current node. The performance metrics include memory bandwidth, memory access latency, the proportion of CPU soft interrupt handling time, and the proportion of CPU idle time; Normalize each performance metric and then perform a weighted sum to obtain the corresponding performance score. The higher the performance score, the better the performance of the corresponding NUMA node.

[0013] The present application performs a load test on the CPU during the deployment phase to obtain the actual load characteristics representing the node performance, adaptively selects the optimal NUMA node, generates a cpuSet, and ensures that the container process threads are bound to the NUMA node with the best performance. In cooperation with the above-mentioned determination of the adaptive memory configuration based on the memory requirements, resource optimization can be completed during the deployment phase, avoiding resource configuration deficiencies, adjustment delays, and interruption risks during the runtime.

[0014] Note: cpuSet is a parameter used to specify the set of CPU cores on which a process or thread can run, and is usually used in a container orchestration system (such as Kubernetes) or a specific application configuration.

[0015] As an implementable manner: After receiving the key parameters input by the user, the current node performs container deployment based on the key parameters, and during the deployment process, transmits the corresponding deployment status and logs back to the central node through the SSH standard stream.

[0016] In this application, the nodes for container deployment will autonomously detect resources locally, dynamically deduce deployment parameters, and complete the deployment independently. At the same time, the deployment status is transmitted back in real-time through the SSH standard stream, realizing the organic combination of centralized unified control and local adaptive deployment of nodes, taking into account both deployment flexibility and controllability, and significantly improving the efficiency and success rate of large-scale container deployment in heterogeneous environments.

[0017] Note: SSH, Secure Shell, is a cryptographic network transport protocol.

[0018] As an implementable manner: The steps for adaptively configuring memory as memory_limit and assigning values to the placeholders corresponding to memory_limit based on the corresponding assignment rules are as follows: Perform load simulation on the memory of the current node to obtain corresponding memory metrics, where the memory metrics include the peak heap memory occupancy and the cache hit rate; Predict the memory requirements corresponding to the current node based on the memory metrics by a preset linear regression model to obtain a first prediction result; Predict the memory requirements corresponding to the current node based on the memory metrics by a preset random forest regression model to obtain a second prediction result; Perform weighted fusion on the first prediction result and the second prediction result to obtain the target memory requirement range; Calculate the corresponding maximum memory limit value based on the target memory requirement range and assign a value to the placeholder corresponding to memory_limit.

[0019] For deployment parameters related to memory requirements, in traditional container deployment, they are often manually configured by humans or configured based on preset default values, and then adjusted based on runtime resource adaptability (such as Kubernetes VPA). This application breaks through the technical prejudice that those skilled in the art believe that memory optimization can only be carried out during the runtime. Through load simulation and model prediction, the corresponding memory requirement range is predicted during the deployment phase and memory optimization is completed based on the memory requirement range, avoiding memory shortages, resource waste, or service interruptions caused by adjustments during the runtime, and improving deployment intelligence and resource utilization.

[0020] Note: memory_limit, memory limit, is used to limit the maximum amount of memory that a process, application, or container can use; Kubernetes VPA, where VPA (Vertical Pod Autoscaler) is an auto-scaling tool for Kubernetes, used to dynamically adjust the resource requests (requests) and limits of containers based on the usage of containers.

[0021] As an implementable manner: The key parameters include service type parameters and / or environment configuration key parameters.

[0022] As an implementable manner: The dynamic template includes: A service template corresponding to the service type parameter, which is used to indicate the corresponding management background interface; A policy template corresponding to the environment configuration key parameter, which is used to indicate the mounting policy or port mapping policy; There are one or more candidate templates that match the key parameters input by the user, and a corresponding dependency graph is generated based on all the matching candidate templates.

[0023] In a second aspect, the present application proposes a containerized deployment tool for a graph database, which is used to perform adaptive deployment on the current node based on the key parameters input by the user, including: A matching unit, which is used to match the key parameters with a preset dynamic template to obtain corresponding candidate templates. Each dynamic template contains placeholders, and the assignment rules and dependency relationships corresponding to each placeholder; A generating unit, which is used to use the placeholders corresponding to the obtained candidate templates as nodes and the dependency relationships as edges to generate a corresponding dependency graph, and the dependency graph is a directed acyclic graph; A derivation unit, which is used to sequentially derive the actual values corresponding to each placeholder according to the corresponding assignment rules based on the dependency graph to generate corresponding deployment parameters; A deployment unit, which is used to perform container deployment on the current node based on the deployment parameters; The dependency graph includes placeholders corresponding to adaptive memory configuration. The derivation unit performs load simulation on the memory of the current node based on the assignment rules corresponding to the placeholders to obtain corresponding memory metrics; and predicts the memory requirements based on the memory metrics to obtain corresponding prediction results; based on the prediction results, generate an adaptive memory configuration corresponding to the current node and assign values to the corresponding placeholders.

[0024] In a third aspect, the present application proposes a containerized deployment system for a graph database, which is used to execute the containerized deployment method described in any one of the above.

[0025] Due to the adoption of the above technical solutions, the present invention has remarkable technical effects: Through the design of dynamic templates and dependency graphs, during the deployment process, the present invention automatically generates corresponding dependency graphs to determine the logical relationships between corresponding placeholders, and automatically generates corresponding parameters according to the dependency graphs and corresponding assignment rules. Compared with the existing solutions based on static template deployment, it can dynamically deduce the required parameters according to the deployment environment, achieve adaptive deployment in different hardware environments, eliminate manual adjustment, reduce manual intervention, improve deployment efficiency, and be applicable to scenarios of single-node deployment and multi-node batch deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0027] Figure 1 is a schematic flowchart of a method for containerized deployment of a graph database in Embodiment 1 of the present invention; Figure 2 is the module connection of a tool for containerized deployment of a graph database in Embodiment 2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The following further elaborates on the present invention in conjunction with embodiments. The following embodiments are explanations of the present invention, and the present invention is not limited to the following embodiments.

[0029] Embodiment 1: A method for containerized deployment of a graph database, which performs adaptive deployment on the current node based on key parameters input by the user, as Figure 1 shown, includes the following steps: S100. Based on the key parameters, match with a preset dynamic template to obtain corresponding candidate templates; In this embodiment, the key parameters include service type parameters and / or environment configuration key parameters.

[0030] The service type parameters include parameters such as admin, graph, etc.; The environment configuration key parameters include parameters such as the data storage path (—home parameter), etc.; Those skilled in the art can set the environment configuration key parameters that need to be configured by the user according to actual needs, and this embodiment does not limit them in detail.

[0031] In this embodiment, the dynamic template includes a service template corresponding to the service type parameter, and also includes a policy template corresponding to the key environmental configuration parameter; Among them, the service template is used to indicate the corresponding management background interface. In practical applications, it can also indicate the data read / write port according to actual needs, and its cache configuration is relatively lightweight; The policy template is used to indicate the mounting policy or port mapping policy. The characteristics of the policy template in this embodiment include default mapping of large-capacity mount points, high cache hit rate priority policy, and log sub-directory storage.

[0032] In practical applications, several dynamic templates are pre-constructed. According to the key parameters required by the dynamic templates, corresponding tags are set for the dynamic templates; after obtaining the key parameters configured by the user, based on the matching of the key parameters with the tags of each dynamic template, for example, based on the service type parameter (such as admin / graph, etc.) to determine the matching service template (such as admin-template, graph-template), and further screen the corresponding policy template based on the key environmental configuration parameters such as data storage path, port, mounting path, etc. Finally, one or more dynamic templates obtained by matching are used as candidate templates.

[0033] S200. Use the placeholder corresponding to the obtained candidate template as a point and the dependency relationship as an edge to generate a corresponding dependency graph, and the dependency graph is a directed acyclic graph; In this embodiment, each preset dynamic template includes placeholders, the assignment rules and dependency relationships corresponding to each placeholder. Among them, the dependency relationship is used to indicate the dependency relationship between placeholders, the placeholder is used to indicate the configuration parameter to be deduced, the assignment rule is used to indicate the way to obtain this placeholder, the dependency relationship is used to indicate the relevant parameters required to obtain this placeholder, and the dependency relationship can indicate the derivation direction; In this embodiment, the obtained one or more candidate templates are combined into a complete template to obtain a corresponding target template; Parse the placeholders and corresponding dependency relationships in the target template, establish a corresponding relationship graph, and generate a directed acyclic graph based on topological sorting to obtain a dependency graph. Each vertex in the obtained dependency graph is a placeholder, and the edge is the corresponding dependency relationship, and the derivation order is indicated by the direction of the edge.

[0034] This embodiment adopts a combination of dynamic templates and dependency graphs, and there is no need to manually synchronize and maintain the dependency graph when the template is extended or the parameters are updated, and the adaptability and decoupling ability are strong.

[0035] S300. Based on the dependency graph, deduce the actual values corresponding to each placeholder in turn according to the corresponding assignment rules, generate corresponding deployment parameters, and perform container deployment on the current node based on the deployment parameters; The specific steps for generating deployment parameters are as follows: S310. Generate a context object based on the dependency graph; S320. Use the placeholder to be derived currently as the target placeholder based on the dependency graph, and obtain the assignment rule corresponding to the target placeholder; S330. Obtain the corresponding parameter value based on the assignment rule and write the parameter value into the context object; S340. Determine whether the derivation of all placeholders is completed; If not, repeat steps S320 and S340; If so, generate the corresponding deployment parameters based on the obtained context object.

[0036] In this embodiment, the types of assignment rules include: Direct assignment: directly obtain the corresponding value based on a preset rule and assign it. For example, the assignment rule for the {logs_mount} placeholder (used to indicate the mount path or storage location of the log file) is to determine whether the user specifies the -mount parameter. When specified, assign based on the value of the specified -mount parameter; otherwise, assign according to the value of the -home parameter.

[0037] Perceived assignment: detect the corresponding value based on a preset rule and assign it. For example, the assignment rule for the {admin_port} placeholder (the port of the diagram management web service) is to detect whether each port is occupied in sequence (starting from the default port). When an unoccupied port is detected, assign based on this port.

[0038] Calculated assignment: perform dynamic context calculation based on a preset rule, obtain the corresponding calculation result, and assign it. For example, the {memory_limit} placeholder predicts the memory requirement based on the corresponding memory metrics and calculates the corresponding parameter value based on the obtained memory requirement.

[0039] For traditional container deployment templates (such as the Kubernetes management tool Helm Chart), which are static template placeholders replaced only by manual filling by users or simple default value overwriting and cannot be dynamically derived according to environmental changes; In this embodiment, through the design of dynamic templates and dependency graphs, during the deployment process, the corresponding dependency graph is automatically generated to determine the logical relationship between the corresponding placeholders, and the corresponding parameters are automatically generated according to the dependency graph and the corresponding assignment rules. Compared with the existing static template-based deployment solutions, it can dynamically derive the required parameters according to the deployment environment, achieve adaptive deployment in different hardware environments, eliminate manual adjustment, reduce manual intervention, improve deployment efficiency, and is applicable to single-node deployment and multi-node batch deployment scenarios.

[0040] As an implementable manner: The generated dependency graph includes placeholders corresponding to adaptive memory configuration. In this embodiment, the placeholder corresponding to adaptive memory configuration is the {memory_limit} placeholder; The steps of assigning values to the placeholders corresponding to adaptive memory configuration based on corresponding assignment rules are as follows: S410. Simulate the memory load of the current node to obtain corresponding memory metrics; Deploy a lightweight simulation load thread on the current node to simulate the actual workload of the graph database and collect corresponding memory metrics. In this embodiment, the memory metrics include the peak occupancy of heap memory and cache hit rate; Those skilled in the art can select key metrics corresponding to memory usage as the memory metrics according to actual needs, such as the peak total memory occupancy, memory page swap-out volume, etc. This embodiment does not limit it in detail.

[0041] Those skilled in the art can perform load simulation according to common memory load scenarios of the graph database, such as simulating a large number of insertions of vertices and edges, complex queries, etc. This is prior art and this specification does not describe it in detail. It is sufficient to be able to simulate the actual workload of the graph database for the application logic layer.

[0042] S420. Predict the memory requirement based on the memory metrics to obtain a corresponding prediction result; Specifically: S421. Use a preset linear regression model to predict the memory requirement corresponding to the current node based on the memory metrics to obtain a first prediction result; The linear regression model can quickly estimate the overall trend; S422. Use a preset random forest regression model to predict the memory requirement corresponding to the current node based on the memory metrics to obtain a second prediction result; The random forest regression model can capture complex non-linear relationships in the load characteristics; S423. Weight and fuse the first prediction result and the second prediction result to obtain a target memory requirement range; Those skilled in the art can set the weights corresponding to the first prediction result and the second prediction result according to actual needs, such as target memory requirement range = 0.7 * first prediction result + 0.3 * second prediction result; In this embodiment, through the joint inference mechanism of the linear regression model and the random forest regression model, the corresponding memory requirement range can be accurately predicted.

[0043] In this embodiment, a linear regression model and a random forest regression model are pre-constructed as corresponding prediction models. The input of each prediction model is the memory metrics after normalization, and the output is the corresponding memory requirement range. Finally, the memory requirement ranges output by the two prediction models are weighted and fused to obtain the prediction result; After those skilled in the art know the input (normalized memory metrics) and output (memory requirement range), they can train the corresponding linear regression model and random forest regression model based on the existing publicly disclosed training methods. Therefore, this embodiment will not elaborate on the specific model training steps.

[0044] Both the linear regression model and the random forest regression model adopted in this embodiment are lightweight modeling (the number of decision trees n_estimators < 50, and the maximum depth of a single decision tree max_depth < 10), ensuring that the inference latency during the deployment phase is less than 10 milliseconds, the resource overhead is extremely small, and it does not affect the performance of the overall deployment process, ensuring that the container has the optimal memory configuration when it starts.

[0045] S430. Generate an adaptive memory configuration corresponding to the current node based on the prediction result.

[0046] In this embodiment, the corresponding maximum memory limit value (Memory Limit) is calculated based on the obtained target memory requirement range and updated to the corresponding {memory_limit} placeholder in the context object, that is, assign a value to the {memory_limit} placeholder.

[0047] When the container starts, the memory - Xmx parameter (the maximum value of the JVM heap memory) and the - Xms parameter (the minimum value of the JVM heap memory) of the Java virtual machine (JVM) will be automatically adjusted according to the {memory_limit} value, that is, the corresponding deployment parameters will be automatically generated; The traditional container deployment method configures the parameters corresponding to the memory by manually configuring resources and relies on Kubernetes VPA (Vertical Pod Autoscaler) to adjust during the running period. This embodiment designs the corresponding placeholder and the corresponding assignment rule, which can accurately configure the memory limit and generate the corresponding deployment parameters during the deployment phase, effectively avoiding memory shortage, resource waste or service interruption caused by adjustment during the running period, and improving the deployment intelligence and resource utilization rate.

[0048] As an implementable manner: The generated dependency graph includes the placeholder corresponding to cpu_bindings. At this time, this containerized deployment method is applied to a multi - node deployment scenario, and the NUMA node for container deployment is bound through the placeholder corresponding to cpu_bindings; In this embodiment, the assignment rules corresponding to the cpu_bindings placeholder include a performance evaluation rule and a deployment decision rule; The steps of assigning values to the placeholders corresponding to cpu_bindings based on the corresponding assignment rules are as follows: S510. Perform a performance evaluation on the current node based on the performance evaluation rule to obtain the performance score corresponding to the current node; Specifically: S511. Simulate the load on the CPU of the current node to obtain the performance metrics corresponding to the current node; The performance metrics include memory bandwidth, memory access latency, the proportion of CPU soft interrupt handling time, and the proportion of CPU idle time; In this embodiment, a lightweight simulation load program is deployed on each NUMA node respectively to simulate the relevant operations of the graph database at the hardware level, so as to measure the performance of the underlying nodes.

[0049] Those skilled in the art can select the performance metrics indicating node performance according to actual needs. In this embodiment, the performance metrics include: Memory bandwidth, with the unit of MB / s. The higher this value, the better the performance; Memory access latency, with the unit of ns. The lower this value, the better the performance; The proportion of CPU soft interrupt handling time, with the unit of %. The lower this value, the better the performance; The proportion of CPU idle time, with the unit of %. The higher this value, the better the performance.

[0050] Those skilled in the art can simulate the load for typical operations involved in the graph database, such as random memory access and memory bandwidth impact. This is prior art and will not be elaborated in detail in this specification. It is only necessary to be able to simulate the operations of the graph database at the hardware level to measure node performance characteristics such as memory bandwidth and access latency.

[0051] S512. After normalizing each performance metric, perform a weighted sum to obtain the corresponding performance score; The higher the performance score, the better the performance of the corresponding NUMA node.

[0052] In this embodiment, each performance metric is normalized to the interval [0, 1) to eliminate the influence of dimensions; Those skilled in the art can set the weighted weights corresponding to each performance metric according to actual needs. For example, in this embodiment, the weighted weight corresponding to memory bandwidth is 0.4, the weighted weight corresponding to memory access latency is 0.3, the weighted weight corresponding to the proportion of CPU idle time is 0.2, and the weighted weight corresponding to the proportion of CPU soft interrupt handling time is 0.1; When it is found that the node load is too high or resource bottlenecks occur during the detection process, the weighted weights corresponding to each performance metric can be dynamically reallocated according to the preset adjustment rules. In this embodiment, the weight deviation is adjusted to the range of ±5%.

[0053] S520. Generate a deployment decision corresponding to the current node based on the deployment decision rule and the performance scores of each NUMA node; In this embodiment, the NUMA nodes for container deployment are determined based on the performance scores. For example, all NUMA nodes are sorted in descending order of performance scores, and several NUMA nodes with the highest performance scores are used as the nodes for container deployment. Based on the selected nodes for container deployment, the deployment strategy of the current node is determined.

[0054] Furthermore, based on the performance scores, the number of idle cores, and the available memory space, a preset number of NUMA nodes are selected as the nodes for container deployment; Specifically: S521. Sort all NUMA nodes in descending order of performance scores to obtain a candidate list; S522. Take the node with the highest performance score in the candidate list as the first candidate node, and filter out the nodes in the candidate list whose performance score differences from the candidate node are less than the preset difference threshold (5%) to obtain the corresponding second candidate nodes; S523. When there are no corresponding second candidate nodes; Take the first candidate node as the node for container deployment, and delete the first candidate node from the candidate list; after completion, go to step S525; S524. When there are corresponding second candidate nodes; Based on the preset screening rules, preferentially select the node with more idle cores and / or larger available memory space from the first candidate node and each second candidate node as the node for container deployment, and delete the corresponding candidate node from the candidate list; after completion, go to step S525; For example: Take the node with the largest number of idle cores among the first candidate node and each second candidate node as the third candidate node; When the number of third candidate nodes is equal to 1, take the third candidate node as the node for container deployment; When the number of third candidate nodes is greater than 1, select the node with the largest available memory space from the third candidate nodes as the node for container deployment.

[0055] S525. Determine whether the number of nodes for container deployment reaches the corresponding preset value; Thus, the end node screening is completed; If not, repeat the above steps from S522 to S525.

[0056] S530. When the deployment decision is to cancel the deployment, the derivation of the dependency graph is ended; That is, when the current node does not belong to the node for container deployment, the deployment task is ended.

[0057] S540. When the deployment decision is to perform container deployment, assign values to the placeholders corresponding to cpu_bindings based on the CPU logical core set of the current node, and continue to derive the actual values corresponding to the subsequent placeholders based on the corresponding dependency graph; That is, when the current node belongs to the node for container deployment, bind the current node and continue with the corresponding deployment task.

[0058] When performing multi-node deployment of a graph database, it is often the case that Kubernetes uniformly distributes a static YAML template (YAML is a human-readable data serialization format), and uses an Ansible playbook (an automated script in YAML format) to execute in batches. However, since the deployment environments of different nodes are different, it is still necessary to manually adjust the deployment parameters for each node according to the actual situation. As the number of nodes to be deployed increases, the deployment process becomes extremely long and cumbersome. In this embodiment, through the design of the value assignment rule for cpu_bindings, the nodes for container deployment are automatically selected, and the corresponding nodes adaptively generate configurations locally, which are compatible with diverse hardware environments and are dynamically derived according to the actual resources of the nodes, eliminating the need for manual secondary adjustment.

[0059] Furthermore, the placeholder also includes CPU topology information, the architecture of the CPU, the number of NUMA nodes, the distribution of CPU cores, and the memory channel mapping relationship corresponding to the CPU; The CPU topology information and the architecture of the CPU are used to determine whether NUMA optimization is supported. For example, in a virtual machine environment, if the socket and vcpus are not reasonably allocated, there may be an unexpected Numa Node; The number of NUMA nodes is used to determine on which nodes to deploy a simulation load program (for node performance evaluation) and determine the distribution range of subsequent load tests; The distribution of CPU cores is used to bind the simulation load program (for node performance evaluation) separately according to NUMA nodes to ensure the independent performance evaluation of each node without cross-contamination; The memory channel mapping relationship corresponding to the CPU is used to accurately attribute the data accessed by the CPU cores to the corresponding NUMA memory when evaluating memory bandwidth and latency in the subsequent stage, ensuring the positioning accuracy of the load test.

[0060] Note: "socket" represents the physical CPU of the server and can be configured as the topology of virtual CPUs; "vcpus" represents the logical CPU unit in the virtualized environment; Numa Node is the NUMA node.

[0061] The above information belongs to the dependent information, and its acquisition methods are all obtained based on the built-in hwloc (Portable Hardware Locality, an open-source tool library for detecting and visualizing hardware topologies). On the premise of knowing the functions of the above information, those skilled in the art can select and utilize the above information according to actual needs using existing technologies, so it will not be elaborated in this specification.

[0062] In summary, the containerized deployment solution proposed in this embodiment significantly reduces the complexity of containerized deployment of the graph database, improves resource utilization efficiency and deployment stability, and solves the problems of complex parameters, insufficient resource awareness, and difficult adaptation to multi-node environments existing in the prior art.

[0063] Embodiment 2: A containerized deployment tool for a graph database, which is used to perform adaptive deployment on the current node based on key parameters input by the user, as Figure 2 shown, including: A matching unit 100, which is used to match with a preset dynamic template based on the key parameters to obtain corresponding candidate templates. Each dynamic template contains placeholders, and the assignment rules and dependency relationships corresponding to each placeholder; A generating unit 200, which is used to use the placeholders corresponding to the obtained candidate templates as points and the dependency relationships as edges to generate a corresponding dependency graph. The dependency graph is a directed acyclic graph; A derivation unit 300, which is used to sequentially derive the actual values corresponding to each placeholder based on the dependency graph according to the corresponding assignment rules to generate corresponding deployment parameters; A deployment unit 400, which is used to perform container deployment on the current node based on the deployment parameters; The dependency graph includes placeholders corresponding to adaptive memory configuration. The derivation unit 300 performs load simulation on the memory of the current node based on the assignment rules corresponding to the placeholder to obtain corresponding memory metrics; and predicts the memory requirements based on the memory metrics to obtain corresponding prediction results; based on the prediction results, generates an adaptive memory configuration corresponding to the current node and assigns values to the corresponding placeholders.

[0064] This embodiment is the device embodiment corresponding to Embodiment 1. For related parts, refer to the partial description of Embodiment 1.

[0065] Embodiment 3. A containerized deployment method for a graph database, used in a single-node deployment scenario, includes the following steps.

[0066] Pre-deploy the deployment tool described in Embodiment 2 to the current node; Obtain the key parameters input by the user; Based on the key parameters, the deployment tool executes the deployment method described in Embodiment 1 to deploy the corresponding graph database to the current node.

[0067] Embodiment 4. A containerized deployment method for a graph database, used in a multi-node deployment scenario, includes the following steps: The central node receives the key parameters input by the user; The central node sends a performance evaluation instruction to each NUMA node, and each NUMA node performs a performance evaluation based on a preset performance evaluation rule to obtain a corresponding performance score; The central node selects several NUMA nodes as target nodes based on the performance scores corresponding to each NUMA node and a preset deployment decision rule; The central node sends the key parameters and the deployment tool described in Embodiment 2 to each target node; The deployment tool, based on the key parameters, executes the deployment method described in Embodiment 1 on the target node where it is located, performs adaptive deployment on the target node where it is located, and during the deployment process, sends the corresponding deployment status and logs back to the central node through the SSH standard stream.

[0068] In this embodiment, the placeholder corresponding to cpu_bindings is directly assigned a value based on the CPU logical core set of the target node where it is located.

[0069] Embodiment 5. A containerized deployment method for a graph database, used in a multi-node deployment scenario, includes the following steps: The central node receives the key parameters input by the user; The central node sends the key parameters and the deployment tool described in Embodiment 2 to each NUMA node; The deployment tool, based on the key parameters, executes the deployment method described in Embodiment 1 on the target node where it is located, makes a deployment decision on the target node where it is located, and cancels the deployment or performs adaptive deployment based on the obtained deployment decision; When the deployment decision is to cancel the deployment, the derivation of the dependency graph ends; When the deployment decision is to perform container deployment, the placeholder corresponding to cpu_bindings is assigned a value based on the CPU logical core set of the current node, and the actual values corresponding to the subsequent placeholders are continued to be derived based on the corresponding dependency graph.

[0070] In this embodiment, the corresponding deployment status and logs during the deployment process are transmitted back to the central node through the SSH standard stream.

[0071] In this embodiment, the dependency graphs generated by each deployment tool based on the key parameters all include placeholders corresponding to cpu_bindings, and the assignment rules corresponding to these placeholders include performance evaluation rules and deployment decision rules. Therefore, during the process of deducing the cpu_bindings placeholders based on the dependency graph, deployment decisions are made, and according to the decision results, the placeholders corresponding to cpu_bindings are assigned values to bind the current node, or the container deployment of the current node is cancelled.

[0072] Since generating context objects based on the corresponding dynamic templates is a lightweight operation and does not consume node resources; and considering costs and business requirements, most graph databases are deployed to 3 or 5 nodes. In such scenarios, the deployment efficiency of the solution in Embodiment 4 that first performs performance evaluation on each NUMA node and then deploys containers to the target node is not much different from the solution in this embodiment that combines performance evaluation and parameter deduction.

[0073] Benefiting from the development of container technology, currently graph databases have successfully solved the compatibility problem on multiple operating systems through containerized deployment. However, due to the complexity of the graph database cluster system itself, performance and functions need to be adjusted and controlled through configuration during the cluster deployment process; especially in the multi-node cluster deployment scenario, the same deployment process needs to be repeatedly executed on each node, even in a container environment; therefore, how to correctly implement the containerization of graph databases and achieve rapid deployment has become a key issue; In the prior art, in addition to the relevant parameters directly assigned based on default values, it is necessary to manually configure the corresponding deployment parameters for each node for container deployment, and send fixed deployment parameters or instructions to the corresponding nodes for containerized deployment; in the multi-node deployment scenario, it is also necessary to manually configure the parameters for each node entering the container deployment, with low efficiency and insufficient adaptability to the operating environment; Referring to the above Embodiment 4 and Embodiment 5, in the present invention, the central node uniformly distributes key parameters (the quantity is much smaller than the deployment parameters) and deployment tools, enabling each node to locally and autonomously detect resources, make intelligent decisions, dynamically deduce deployment parameters, and independently complete the deployment. At the same time, the deployment status is transmitted back in real time through the SSH standard stream, realizing the organic combination of central unified control and local adaptive deployment of nodes, taking into account both deployment flexibility and controllability, and significantly improving the efficiency and success rate of large-scale container deployment in heterogeneous environments; it can be widely applied to the containerized deployment scenario of large-scale graph database systems, and is particularly suitable for enterprise-level application scenarios with heterogeneous environments and frequent dynamic changes of resources.

[0074] Example 6. A containerized deployment system for a graph database is used to execute the containerized deployment method described in any one of Examples 3 to 5.

[0075] The embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0076] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the present invention can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] The present invention is described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0078] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal device provide steps for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0080] It should be noted that: As used in the specification, the phrase "an embodiment" or "embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, the phrases "an embodiment" or "embodiments" that appear throughout the specification do not necessarily all refer to the same embodiment.

[0081] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention.

[0082] In addition, it should be noted that the specific embodiments described in this specification may have different shapes, names, etc. for their components. Any equivalent or simple changes made to the structure, features, and principles described according to the inventive concept of the present invention are included within the scope of protection of the present invention. Those skilled in the art to which the present invention pertains can make various modifications, supplements, or use similar means of substitution to the specific embodiments described, as long as they do not deviate from the structure of the present invention or exceed the scope defined by this claims, they should fall within the scope of protection of the present invention.

Claims

1. A method for containerized deployment of a graph database, characterized in that, Based on the key parameters input by the user, perform adaptive deployment at the current node, including the following steps: Based on the key parameters, match with the preset dynamic templates to obtain corresponding candidate templates. Each dynamic template includes placeholders, and the assignment rules and dependencies corresponding to each placeholder; Take the placeholders corresponding to the obtained candidate templates as points and the dependencies as edges to generate a corresponding dependency graph. The dependency graph is a directed acyclic graph; Based on the dependency graph, deduce the actual values corresponding to each placeholder in sequence according to the corresponding assignment rules, and generate corresponding deployment parameters; Perform container deployment at the current node based on the deployment parameters; The dependency graph includes placeholders corresponding to adaptive memory configuration. The step of assigning values to the placeholders corresponding to adaptive memory configuration according to the corresponding assignment rules is: Perform load simulation on the memory of the current node to obtain corresponding memory metrics; Predict the memory requirements based on the memory metrics to obtain corresponding prediction results; Based on the prediction results, generate an adaptive memory configuration corresponding to the current node.

2. The containerized deployment method of the graph database according to claim 1, wherein When performing multi-node deployment, it also includes the step of selecting NUMA nodes for container deployment. Specifically: Each NUMA node performs performance evaluation based on the preset performance evaluation rules to obtain corresponding performance scores; Based on the performance scores corresponding to each NUMA node and the preset deployment decision rules, select several NUMA nodes as target nodes; The central node distributes the key parameters input by the user to each target node, and each target node performs adaptive deployment based on the key parameters.

3. The containerized deployment method of the graph database according to claim 1, characterized in that When performing multi-node deployment, the central node distributes the key parameters input by the user to each NUMA node. Each NUMA node automatically makes a deployment decision based on the key parameters, and cancels the deployment or performs adaptive deployment based on the obtained deployment decision; Among them, based on the key parameters, a dependency graph corresponding to the current node is generated. The dependency graph includes placeholders corresponding to cpu_bindings, and the assignment rules corresponding to this placeholder include performance evaluation rules and deployment decision rules; When assigning values to the placeholders corresponding to cpu_bindings according to the corresponding assignment rules: Perform performance evaluation on the current node based on the performance evaluation rules to obtain the performance score corresponding to the current node; Based on the deployment decision rules and the performance scores corresponding to each NUMA node, generate a deployment decision corresponding to the current node; When the deployment decision is to cancel the deployment, end the derivation of the dependency graph; When the deployment decision is to perform container deployment, assign values to the placeholders corresponding to cpu_bindings based on the CPU logical core set of the current node, and continue to deduce the actual values corresponding to the subsequent placeholders based on the corresponding dependency graph.

4. The containerized deployment method of the graph database according to claim 2 or 3, characterized in that, The step of the current node performing performance evaluation based on the performance evaluation rules includes: Perform load simulation on the CPU of the current node to obtain the performance metrics corresponding to the current node. The performance metrics include memory bandwidth, memory access latency, the proportion of CPU soft interrupt handling time, and the proportion of CPU idle time; After normalizing each performance indicator and performing weighted summation, a corresponding performance score is obtained. The higher the performance score, the better the performance of the corresponding NUMA node.

5. The method for containerized deployment of a graph database according to claim 2 or 3, characterized in that: After the current node receives the key parameters input by the user, container deployment is performed based on the key parameters, and the corresponding deployment status and logs are transmitted back to the central node through the SSH standard stream during the deployment process.

6. The method for containerized deployment of a graph database according to any one of claims 1 to 3, characterized in that: The step of adaptively configuring memory to memory_limit and assigning values to the placeholders corresponding to memory_limit based on the corresponding assignment rules is as follows: Perform a load simulation on the memory of the current node to obtain corresponding memory metrics, where the memory metrics include the peak heap memory occupancy and the cache hit rate; Predict the memory requirements corresponding to the current node based on the memory metrics by a preset linear regression model to obtain a first prediction result; Predict the memory requirements corresponding to the current node based on the memory metrics by a preset random forest regression model to obtain a second prediction result; Perform weighted fusion on the first prediction result and the second prediction result to obtain a target memory requirement range; Calculate the corresponding maximum memory limit value based on the target memory requirement range and assign a value to the placeholder corresponding to memory_limit.

7. The method for containerized deployment of a graph database according to any one of claims 1 to 3, characterized in that: The key parameters include service type parameters and / or environment configuration key parameters.

8. The method for containerized deployment of a graph database according to claim 7, characterized in that: The dynamic template includes: A service template corresponding to the service type parameter, and the service template is used to indicate the corresponding management background interface; A policy template corresponding to the environment configuration key parameter, and the policy template is used to indicate the mounting policy or the port mapping policy; One or more candidate templates that match the key parameters input by the user are used to generate a corresponding dependency graph based on all the matching candidate templates.

9. A containerized deployment tool for a graph database, characterized in that, For adaptive deployment on the current node based on the key parameters input by the user, it includes: A matching unit for matching with a preset dynamic template based on the key parameters to obtain corresponding candidate templates. Each dynamic template contains placeholders, and each placeholder has a corresponding assignment rule and dependency relationship; A generating unit for using the placeholders corresponding to the obtained candidate templates as nodes and the dependency relationships as edges to generate a corresponding dependency graph, and the dependency graph is a directed acyclic graph; A derivation unit for sequentially deriving the actual values corresponding to each placeholder based on the dependency graph according to the corresponding assignment rules to generate corresponding deployment parameters; A deployment unit for performing container deployment on the current node based on the deployment parameters. The dependency graph includes placeholders corresponding to adaptive memory configurations. The derivation unit performs load simulation on the memory of the current node based on the assignment rules corresponding to the placeholders to obtain corresponding memory metrics; predicts the memory requirements based on the memory metrics to obtain corresponding prediction results; generates an adaptive memory configuration corresponding to the current node based on the prediction results, and assigns values to the corresponding placeholders.

10. A containerized deployment system for a graph database, characterized in that, For executing the containerized deployment method according to any one of claims 1 to 8.

Images