Containerized deployment methods, tools, and systems for graph databases

Abstract

The present invention discloses a containerized deployment method, tool and system for a graph database, wherein the method is used to perform adaptive deployment at the current node based on key parameters input by the user, and includes the following steps: based on the key parameters, matching with preset dynamic templates to obtain corresponding candidate templates, each dynamic template contains placeholders, assignment rules and dependency relationships corresponding to each placeholder; using the corresponding placeholders in the obtained candidate templates as points and the dependency relationships as edges to generate a corresponding dependency graph, which is a directed acyclic graph; based on the dependency graph, deriving the actual values ​​corresponding to each placeholder in turn according to the corresponding assignment rules, and generating corresponding deployment parameters; and performing container deployment at the current node based on the deployment parameters. Compared with existing static templates based on manual adjustment or simple default value overwriting, the present invention automatically derives the required parameters through the design of dynamic templates and dependency graphs, effectively improving deployment efficiency.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and in particular to a containerized deployment method, tool, and system for a graph database. Background Art

[0002] Traditional graph database deployment tools, such as installation scripts and deployment control panels, rely on manual filling of a large number of configuration items. In addition, configurations need to be manually adjusted based on actual conditions in different production environments. This makes it difficult to quickly adapt to complex environments, and leads to complex deployment, prone to errors, and difficult maintenance. Summary of the Invention

[0003] In response to the shortcomings of the prior art in which graph database deployment requires manual filling of a large number of configuration items, resulting in low efficiency, prone to errors, and difficult maintenance, the present invention provides a containerized deployment method, tool, and system for graph databases.

[0004] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0005] First, this application proposes a containerized deployment method for a graph database, which performs adaptive deployment on the current node based on key parameters input by the user, including the following steps:

[0006] Based on the key parameters, matching is performed with preset dynamic templates to obtain corresponding candidate templates, each dynamic template including placeholders, assignment rules and dependency relationships corresponding to each placeholder;

[0007] The corresponding placeholders in the obtained candidate templates are used as points and the dependency relationships are used as edges to generate a corresponding dependency graph, which is a directed acyclic graph;

[0008] Based on the dependency graph, the actual value corresponding to each placeholder is deduced in sequence according to the corresponding assignment rules, and the corresponding deployment parameters are generated;

[0009] Deploy the container on the current node based on the deployment parameters;

[0010] The dependency graph includes placeholders corresponding to the adaptive memory configuration, and the steps of assigning values ​​to the placeholders corresponding to the adaptive memory configuration based on corresponding assignment rules are as follows:

[0011] Simulate the memory load of the current node to obtain the corresponding memory indicators;

[0012] Predicting memory demand based on the memory indicator to obtain corresponding prediction results;

[0013] Based on the prediction result, an adaptive memory configuration corresponding to the current node is generated.

[0014] Through the design of dynamic templates and dependency graphs, this application allows users to configure only a small number of key parameters, and other configuration parameters can be automatically derived based on the matching dynamic template, reducing manual intervention and improving deployment efficiency.

[0015] Compared with existing static templates based on manual adjustment or simple default value overwriting, the dynamic template provided by this application has corresponding assignment rules embedded in it. For some parameters corresponding to memory requirements, it can automatically predict memory requirements based on the assignment rules and perform adaptive configuration based on the prediction results.

[0016] As an implementable method, when performing multi-node deployment, the step of selecting a NUMA node for container deployment is also included, specifically:

[0017] Each NUMA node performs performance evaluation based on preset performance evaluation rules and obtains a corresponding performance score;

[0018] Select several NUMA nodes as target nodes based on the performance scores of each NUMA node and the preset deployment decision rules;

[0019] The central node distributes the key parameters input by the user to each target node, and each target node performs adaptive deployment based on the key parameters.

[0020] In a multi-node deployment scenario, unlike traditional container deployments where NUMA binding is manually specified via the -cpu command, this application evaluates the performance of each NUMA node and selects the best-performing NUMA node for binding, maximizing local memory access efficiency and reducing cross-node communication latency.

[0021] Note: NUMA, Non-Uniform Memory Access, non-uniform memory access.

[0022] As an implementation method, when performing multi-node deployment, the central node distributes the key parameters input by the user to each NUMA node, and each NUMA node automatically makes a deployment decision based on the key parameters, and cancels the deployment or performs an adaptive deployment based on the resulting deployment decision;

[0023] wherein, based on the key parameters, a dependency graph corresponding to the current node is generated, wherein the dependency graph includes a placeholder corresponding to cpu_bindings, and the assignment rules corresponding to the placeholder include performance evaluation rules and deployment decision rules;

[0024] When assigning values ​​to the placeholders corresponding to cpu_bindings based on the corresponding assignment rules:

[0025] Performing a performance evaluation on the current node based on the performance evaluation rules to obtain a performance score corresponding to the current node;

[0026] Generate a deployment decision corresponding to the current node based on the deployment decision rule and the performance score corresponding to each NUMA node;

[0027] When the deployment decision is to cancel the deployment, ending the derivation of the dependency graph;

[0028] When the deployment decision is to deploy a container, the placeholder corresponding to cpu_bindings is assigned a value based on the CPU logical core set of the current node, and the actual value corresponding to the subsequent placeholder is further derived based on the corresponding dependency graph.

[0029] As above, in the multi-node deployment scenario, this application evaluates the performance of each NUMA node and selects the NUMA node with the best performance for binding. This solution uses the performance evaluation rules and deployment decision rules as the assignment rules corresponding to the cpu_bindings placeholder, so that when the cpu_bindings placeholder is deduced, an intelligent decision is made based on the corresponding assignment rules whether to deploy the container on the current node.

[0030] Note: cpu_bindings is a parameter used to bind a process or thread to a specific CPU core.

[0031] As an implementable method, the step of performing performance evaluation on the current node based on the performance evaluation rule includes:

[0032] Perform load simulation on the CPU of the current node to obtain performance indicators corresponding to the current node, including memory bandwidth, memory access latency, CPU soft interrupt processing time ratio, and CPU idle ratio;

[0033] Each performance indicator is normalized and then weighted and summed to obtain a corresponding performance score. A higher performance score indicates better performance of the corresponding NUMA node.

[0034] This application performs load testing on the CPU during the deployment phase to obtain actual load characteristics that characterize node performance, adaptively selects the optimal NUMA node, generates cpuSet, and ensures that the container process threads are bound to the NUMA node with the best performance. Combined with the above-mentioned adaptive memory configuration based on memory requirements, resource optimization can be completed during the deployment phase, avoiding insufficient resource configuration, adjustment delays, and interruption risks during runtime.

[0035] Note: cpuSet is a parameter used to specify the set of CPU cores on which a process or thread can run. It is typically used in container orchestration systems (such as Kubernetes) or specific application configurations.

[0036] As an implementable approach:

[0037] After receiving the key parameters input by the user, the current node deploys the container based on the key parameters, and transmits the corresponding deployment status and logs back to the central node via the SSH standard stream during the deployment process.

[0038] In this application, the nodes performing container deployment will autonomously detect resources locally, dynamically derive deployment parameters and independently complete deployment. At the same time, the deployment status is transmitted back in real time via the SSH standard stream, realizing the organic combination of centralized unified control and local adaptive deployment of nodes, taking into account deployment flexibility and controllability, and significantly improving the efficiency and success rate of large-scale container deployment in heterogeneous environments.

[0039] Note: SSH, Secure Shell, is an encrypted network transmission protocol.

[0040] As an implementable approach:

[0041] The adaptive memory configuration is memory_limit. The steps to assign a value to the placeholder corresponding to memory_limit based on the corresponding assignment rules are as follows:

[0042] Perform a load simulation on the memory of the current node to obtain corresponding memory indicators, including peak heap memory usage and cache hit rate;

[0043] Predicting the memory demand corresponding to the current node based on the memory indicator using a preset linear regression model to obtain a first prediction result;

[0044] Predicting the memory demand corresponding to the current node based on the memory indicator using a preset random forest regression model to obtain a second prediction result;

[0045] Performing weighted fusion on the first prediction result and the second prediction result to obtain a target memory requirement range;

[0046] A corresponding maximum memory limit value is calculated based on the target memory requirement range, and a value is assigned to the placeholder corresponding to memory_limit.

[0047] Deployment parameters related to memory requirements are often manually configured in traditional container deployments, or configured based on preset default values, and subsequently adjusted based on runtime resource adaptation (such as Kubernetes VPA). This application breaks through the technical prejudice of technicians in this field that memory optimization can only be performed at runtime. Through load simulation and model prediction, the corresponding memory requirement range is predicted during the deployment phase and memory optimization is completed based on the memory requirement range, avoiding insufficient memory, resource waste, or business interruption caused by adjustments at runtime, and improving deployment intelligence and resource utilization.

[0048] Note:

[0049] memory_limit, memory limit, used to limit the maximum amount of memory that a process, application, or container can use;

[0050] Kubernetes VPA, VPA (Vertical Pod Autoscaler) is an automatic scaling tool for Kubernetes that is used to dynamically adjust container resource requests (requests) and limits (limits) based on container usage.

[0051] As an implementable approach:

[0052] The key parameters include service type parameters and / or environment configuration key parameters.

[0053] As an implementable approach:

[0054] Dynamic templates include:

[0055] A service template corresponding to the service type parameter, the service template being used to indicate a corresponding management backend interface;

[0056] a policy template corresponding to the key environment configuration parameter, the policy template being used to indicate a mounting policy or a port mapping policy;

[0057] There are one or more candidate templates that match the key parameters input by the user, and a corresponding dependency graph is generated based on all matching candidate templates.

[0058] Secondly, this application proposes a containerized deployment tool for graph databases, which is used to perform adaptive deployment on the current node based on key parameters input by the user, including:

[0059] A matching unit, configured to match the key parameters with preset dynamic templates to obtain corresponding candidate templates, each dynamic template including placeholders, assignment rules and dependency relationships corresponding to each placeholder;

[0060] A generating unit, configured to use the corresponding placeholders in the obtained candidate templates as points and the dependency relationships as edges to generate a corresponding dependency graph, wherein the dependency graph is a directed acyclic graph;

[0061] A derivation unit, configured to derive the actual value corresponding to each placeholder in turn according to the corresponding assignment rules based on the dependency graph, and generate corresponding deployment parameters;

[0062] A deployment unit, configured to deploy a container on a current node based on the deployment parameters;

[0063] The dependency graph includes placeholders corresponding to the adaptive memory configuration. The derivation unit simulates the memory load of the current node based on the assignment rules corresponding to the placeholder to obtain corresponding memory indicators; and predicts memory requirements based on the memory indicators to obtain corresponding prediction results; based on the prediction results, generates an adaptive memory configuration corresponding to the current node and assigns values ​​to the corresponding placeholders.

[0064] On the third aspect, the present application proposes a containerized deployment system for a graph database, which is used to execute any of the containerized deployment methods described above.

[0065] The present invention has significant technical effects due to the adoption of the above technical solutions:

[0066] Through the design of dynamic templates and dependency graphs, the present invention automatically generates corresponding dependency graphs during the deployment process to determine the logical relationship between corresponding placeholders, and automatically generates corresponding parameters according to the dependency graphs and corresponding assignment rules. Compared with the existing solutions based on static template deployment, the present invention can dynamically derive the required parameters according to the deployment environment, realize adaptive deployment in different hardware environments, eliminate manual adjustments, reduce manual intervention, and improve deployment efficiency. It is applicable to scenarios of single-node deployment and multi-node batch deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 This is a flow chart of a containerized deployment method for a graph database in Example 1 of the present invention;

[0069] Figure 2 It is a module connection of a containerized deployment tool for a graph database in Example 2. DETAILED DESCRIPTION

[0070] The present invention will be further described in detail below with reference to the examples. The following examples are intended to explain the present invention but the present invention is not limited to the following examples.

[0071] Example 1: A containerized deployment method for a graph database is implemented by adaptively deploying the graph database on the current node based on key parameters input by the user. Figure 1 As shown, the following steps are included:

[0072] S100, based on the key parameters, matching with the preset dynamic template to obtain the corresponding candidate template;

[0073] In this embodiment, the key parameters include service type parameters and / or environment configuration key parameters.

[0074] The service type parameters include parameters such as admin, graph, etc.;

[0075] The key parameters of the environment configuration include parameters such as the data storage path (-home parameter);

[0076] Those skilled in the art may set the key environment configuration parameters that need to be configured by the user according to actual needs, and they are not limited in detail in this embodiment.

[0077] In this embodiment, the dynamic template includes a service template corresponding to the service type parameter and a policy template corresponding to the environment configuration key parameter;

[0078] The service template is used to indicate the corresponding management backend interface. In actual applications, it can also indicate the data read and write ports according to actual needs. Its cache configuration is relatively lightweight.

[0079] The policy template is used to indicate a mount policy or a port mapping policy. The features of the policy template in this embodiment include default mapping of large-capacity mount points, a high cache hit rate priority policy, and log directory storage.

[0080] In actual applications, several dynamic templates are pre-built and corresponding labels are set for the dynamic templates based on the key parameters required by the dynamic templates. After obtaining the key parameters configured by the user, the key parameters are matched with the labels of each dynamic template. For example, the matching service template (such as admin-template and graph-template) is determined based on the service type parameters (such as admin / graph, etc.). The corresponding policy templates are further filtered based on the key configuration parameters of the environment such as the data storage path, port, and mount path. Finally, one or more dynamic templates that are matched are used as candidate templates.

[0081] S200, using the corresponding placeholders in the obtained candidate templates as points and the dependency relationships as edges to generate a corresponding dependency graph, wherein the dependency graph is a directed acyclic graph;

[0082] In this embodiment, each preset dynamic template includes a placeholder, an assignment rule corresponding to each placeholder, and a dependency relationship. The dependency relationship indicates the dependency relationship between the placeholders. The placeholder indicates the configuration parameter to be derived. The assignment rule indicates the method for obtaining the placeholder. The dependency relationship indicates the relevant parameters required to obtain the placeholder. The dependency relationship can indicate the derivation direction.

[0083] In this embodiment, the obtained one or more candidate templates are combined into a complete template to obtain a corresponding target template;

[0084] The placeholders and corresponding dependency relationships in the target template are parsed, the corresponding relationship graph is established, and a directed acyclic graph is generated based on topological sorting to obtain a dependency graph. The vertices in the resulting dependency graph are placeholders, and the edges are corresponding dependency relationships. The direction of the edges indicates the derivation order.

[0085] This embodiment adopts a combination of dynamic templates and dependency graphs, which eliminates the need for manual synchronization and maintenance of dependency graphs when templates are expanded or parameters are updated, and has strong adaptability and decoupling capabilities.

[0086] S300: Based on the dependency graph, derive the actual value corresponding to each placeholder in sequence according to the corresponding assignment rule, generate corresponding deployment parameters, and deploy the container on the current node based on the deployment parameters;

[0087] The specific steps to generate deployment parameters are:

[0088] S310, generating a context object based on the dependency graph;

[0089] S320: Based on the dependency graph, the current placeholder to be derived is used as the target placeholder, and an assignment rule corresponding to the target placeholder is obtained;

[0090] S330: Obtain corresponding parameter values ​​based on the assignment rule and write the parameter values ​​into the context object;

[0091] S340: Determine whether the derivation of all placeholders is completed;

[0092] If not, repeat steps S320 and S340;

[0093] If so, corresponding deployment parameters are generated based on the obtained context object.

[0094] The types of assignment rules in this embodiment include:

[0095] Direct assignment: directly obtains and assigns corresponding values ​​based on preset rules. For example, the assignment rule for the {logs_mount} placeholder (used to indicate the mount path or storage location of log files) is to determine whether the user specifies the --mount parameter. If specified, the value is assigned based on the specified --mount parameter value; otherwise, the value is assigned based on the value of the --home parameter.

[0096] Perception assignment: detect and assign corresponding values ​​based on preset rules. For example, the assignment rule for the {admin_port} placeholder (the port of the image management web service) is to detect whether each port is occupied (starting from the default port) in turn. When an unoccupied port is detected, assign a value based on the port.

[0097] Calculation assignment: Perform contextual dynamic calculations based on preset rules to obtain corresponding calculation results and assign values. For example, the {memory_limit} placeholder predicts memory requirements based on corresponding memory indicators and obtains the corresponding parameter value based on the obtained memory requirements.

[0098] Traditional container deployment templates (such as the Kubernetes management tool Helm Chart) are static template placeholders that can only be replaced manually by users or simply overwritten with default values, and cannot be dynamically derived based on environmental changes.

[0099] Through the design of dynamic templates and dependency graphs, this embodiment automatically generates a corresponding dependency graph during the deployment process to determine the logical relationship between corresponding placeholders, and automatically generates corresponding parameters according to the dependency graph and corresponding assignment rules. Compared with the existing solution based on static template deployment, it can dynamically derive the required parameters according to the deployment environment, realize adaptive deployment in different hardware environments, eliminate manual adjustments, reduce manual intervention, and improve deployment efficiency. It is suitable for single-node deployment and multi-node batch deployment scenarios.

[0100] As an implementable approach:

[0101] The generated dependency graph includes a placeholder corresponding to the adaptive memory configuration. In this embodiment, the placeholder corresponding to the adaptive memory configuration is the {memory_limit} placeholder;

[0102] The steps for assigning values ​​to the placeholders corresponding to the adaptive memory configuration based on the corresponding assignment rules are as follows:

[0103] S410, performing a load simulation on the memory of the current node to obtain corresponding memory indicators;

[0104] Deploy a lightweight simulated load thread on the current node to simulate the actual workload of the graph database and collect corresponding memory metrics. In this embodiment, the memory metrics include peak heap memory usage and cache hit rate.

[0105] Those skilled in the art may select key indicators corresponding to memory usage as the memory indicators according to actual needs, such as the peak value of total memory usage, the amount of memory page swapped out, etc., which are not limited in detail in this embodiment.

[0106] Those skilled in the art can perform load simulation based on common memory load scenarios of graph databases, such as simulating a large number of inserted points and edges, complex queries, etc. This is a prior art and this specification will not go into detail about it. It is sufficient to simulate the actual workload of the graph database for the application logic layer.

[0107] S420: Predicting memory demand based on the memory indicator to obtain a corresponding prediction result;

[0108] Specifically:

[0109] S421: Predicting the memory demand corresponding to the current node based on the memory indicator using a preset linear regression model to obtain a first prediction result;

[0110] Linear regression models can quickly estimate overall trends;

[0111] S422: Predicting the memory demand corresponding to the current node based on the memory indicator using a preset random forest regression model to obtain a second prediction result;

[0112] The random forest regression model can capture the complex nonlinear relationships in load characteristics;

[0113] S423: Perform weighted fusion on the first prediction result and the second prediction result to obtain a target memory requirement range;

[0114] Those skilled in the art can set the weights corresponding to the first prediction result and the second prediction result according to actual needs, such as target memory requirement range = 0.7 first prediction result + 0.3 second prediction result;

[0115] This embodiment uses a joint inference mechanism of a linear regression model and a random forest regression model to accurately predict the corresponding memory requirement range.

[0116] In this embodiment, a linear regression model and a random forest regression model are pre-built as corresponding prediction models. The input of each prediction model is the normalized memory index, and the output is the corresponding memory requirement range. Finally, the memory requirement ranges output by the two prediction models are weighted and fused to obtain the prediction result.

[0117] After knowing the input (normalized memory index) and output (memory requirement range), those skilled in the art can train the corresponding linear regression model and random forest regression model based on the existing publicly available training methods. Therefore, this embodiment does not elaborate on the specific model training steps.

[0118] The linear regression model and random forest regression model used in this embodiment are both lightweight models (the number of decision trees n_estimators < 50, and the maximum growth depth of a single decision tree max_depth < 10), ensuring that the inference delay in the deployment phase is less than 10 milliseconds, with extremely low resource overhead, without affecting the performance of the overall deployment process, and ensuring that the container has the optimal memory configuration when it starts.

[0119] S430. Generate an adaptive memory configuration corresponding to the current node based on the prediction result.

[0120] In this embodiment, a corresponding maximum memory limit value (Memory Limit) is calculated based on the obtained target memory requirement range, and is updated to the corresponding {memory_limit} placeholder in the context object, that is, a value is assigned to the {memory_limit} placeholder.

[0121] When the container starts, the Java Virtual Machine (JVM) memory -Xmx parameter (maximum JVM heap memory) and -Xms parameter (minimum JVM heap memory) are automatically adjusted based on the {memory_limit} value. That is, the corresponding deployment parameters are automatically generated.

[0122] Traditional container deployment methods configure memory parameters by manually configuring resources and relying on Kubernetes VPA (Vertical Pod Autoscaling) for adjustments during runtime. This embodiment designs corresponding placeholders and corresponding assignment rules to accurately configure and set memory limits and generate corresponding deployment parameters during the deployment phase. This effectively avoids insufficient memory, resource waste, or business interruptions caused by adjustments during runtime, thereby improving deployment intelligence and resource utilization.

[0123] As an implementable approach:

[0124] The generated dependency graph includes the placeholder corresponding to cpu_bindings. In this case, the containerized deployment method is applied to multi-node deployment scenarios, and the NUMA nodes for container deployment are bound through the placeholder corresponding to cpu_bindings.

[0125] In this embodiment, the assignment rules corresponding to the cpu_bindings placeholder include performance evaluation rules and deployment decision rules;

[0126] The steps to assign values ​​to the placeholders corresponding to cpu_bindings based on the corresponding assignment rules are:

[0127] S510: Perform a performance evaluation on the current node based on the performance evaluation rule to obtain a performance score corresponding to the current node;

[0128] Specifically:

[0129] S511. Perform load simulation on the CPU of the current node to obtain the performance index corresponding to the current node;

[0130] The performance indicators include memory bandwidth, memory access latency, CPU soft interrupt processing time ratio, and CPU idle time ratio;

[0131] In this embodiment, a lightweight simulation load program is deployed on each NUMA node to simulate the relevant operations of the graph database at the hardware bottom layer to achieve performance measurement of the underlying nodes.

[0132] Those skilled in the art may select performance indicators indicating node performance according to actual needs. In this embodiment, the performance indicators include:

[0133] Memory bandwidth, in MB / s. A higher value indicates better performance.

[0134] Memory access latency, in ns. A lower value indicates better performance.

[0135] The percentage of CPU soft interrupt processing time, in %. The lower the value, the better the performance.

[0136] CPU idle ratio (in %). A higher value indicates better performance.

[0137] Those skilled in the art can perform load simulation on typical operations involved in graph databases, such as random memory access, memory bandwidth impact, etc. This is a prior art and this specification will not go into detail about it. It is sufficient to simulate the operations of the graph database on the underlying hardware to measure node performance characteristics such as memory bandwidth and access latency.

[0138] S512, normalizing each performance indicator and performing weighted summation to obtain a corresponding performance score;

[0139] A higher performance score indicates better performance of the corresponding NUMA node.

[0140] In this embodiment, each performance index is normalized to the interval [0, 1) to eliminate the dimension effect;

[0141] Those skilled in the art can set the weighted weights corresponding to the various performance indicators according to actual needs. For example, in this embodiment, the weighted weight corresponding to the memory bandwidth is 0.4, the weighted weight corresponding to the memory access latency is 0.3, the weighted weight corresponding to the CPU idle ratio is 0.2, and the weighted weight corresponding to the CPU soft interrupt processing time ratio is 0.1.

[0142] When it is found during the detection process that the node load is too high or a resource bottleneck occurs, the weighted weights corresponding to each performance indicator can be dynamically reallocated according to the preset adjustment rules. In this embodiment, the weight deviation is adjusted to a range of ±5%.

[0143] S520: Generate a deployment decision corresponding to the current node based on the deployment decision rule and the performance score corresponding to each NUMA node;

[0144] In this embodiment, the NUMA node for container deployment is determined based on the performance score. For example, all NUMA nodes are sorted in descending order according to the performance score, and several NUMA nodes with the highest performance scores are used as nodes for container deployment. The deployment strategy of the current node is determined based on the selected node for container deployment.

[0145] Furthermore, based on the performance score, number of free cores, and available memory space, a preset number of NUMA nodes are selected as nodes for container deployment;

[0146] Specifically:

[0147] S521. Sort all NUMA nodes in descending order of performance scores to obtain a candidate list.

[0148] S522: select the node with the highest performance score in the candidate list as the first candidate node, and select nodes in the candidate list whose performance score difference with the candidate node is less than a preset difference threshold (5%) to obtain the corresponding second candidate node;

[0149] S523: When there is no corresponding second candidate node;

[0150] The first candidate node is used as a node for container deployment, and the first candidate node is deleted from the candidate list; after completion, the process proceeds to step S525;

[0151] S524: When there is a corresponding second candidate node;

[0152] Based on the preset screening rules, from the first candidate node and each second candidate node, the node with more idle cores and / or larger available memory space is preferentially selected as the node for container deployment, and the corresponding candidate node is deleted from the candidate list; after completion, the process proceeds to step S525;

[0153] For example:

[0154] The node with the largest number of idle cores among the first candidate node and the second candidate nodes is selected as the third candidate node;

[0155] When the number of the third candidate node is equal to 1, the third candidate node is used as the node for container deployment;

[0156] When the number of third candidate nodes is greater than 1, a node with the largest available memory space is selected from the third candidate nodes as the node for container deployment.

[0157] S525: Determine whether the number of nodes for container deployment reaches a corresponding preset value;

[0158] If yes, end node screening;

[0159] If not, repeat the above steps S522 to S525.

[0160] S530: When the deployment decision is to cancel the deployment, end the derivation of the dependency graph;

[0161] That is, when the current node is not a node for container deployment, the deployment task is terminated.

[0162] S540: When the deployment decision is to deploy a container, assign a value to the placeholder corresponding to cpu_bindings based on the CPU logical core set of the current node, and continue to derive actual values ​​corresponding to subsequent placeholders based on the corresponding dependency graph;

[0163] That is, when the current node is a node for container deployment, bind the current node and continue the corresponding deployment task.

[0164] When deploying graph databases on multiple nodes, Kubernetes often uniformly distributes static YAML templates (YAML is a human-readable data serialization format) and uses Ansible playbooks (automated scripts in YAML format) for batch execution. However, the deployment environments of different nodes are different, and manual adjustment of deployment parameters for each node is required based on actual conditions. As the number of nodes to be deployed increases, the deployment process becomes very lengthy and cumbersome. This embodiment automatically selects nodes for container deployment through the design of cpu_bindings assignment rules, and enables the corresponding nodes to generate configurations locally and adaptively. This is compatible with diverse hardware environments and is dynamically derived based on the actual resources of the nodes, eliminating the need for manual secondary adjustments.

[0165] Furthermore, the placeholder also includes CPU topology information and CPU architecture, number of NUMA nodes, distribution of CPU cores, and memory channel mapping relationship corresponding to the CPU;

[0166] CPU topology information and CPU architecture are used to determine whether NUMA optimization is supported. For example, in a virtual machine environment, improper allocation of sockets and vcpus may result in an unexpected number of Numa nodes.

[0167] The number of NUMA nodes is used to determine on which nodes to deploy the simulated load program (for node performance evaluation) and to determine the distribution range of subsequent load tests;

[0168] The distribution of CPU cores is used to bind simulated load balancing programs (used for node performance evaluation) to NUMA nodes, ensuring independent performance evaluation of each node without cross-contamination.

[0169] The memory channel mapping relationship corresponding to the CPU is used to accurately attribute the data accessed by the CPU core to the corresponding NUMA memory when subsequently evaluating memory bandwidth and latency, ensuring the positioning accuracy of the load test.

[0170] Note:

[0171] The socket represents the physical CPU of the server and can be configured as a virtual CPU topology;

[0172] vcpus represents the logical CPU unit in a virtualized environment;

[0173] Numa Node is a NUMA node.

[0174] The above information is dependent information and is obtained based on the built-in hwloc (Portable Hardware Locality, an open source tool library for detecting and visualizing hardware topology). Knowing the role of this information, those skilled in the art can select and use this information based on actual needs using existing technologies. Therefore, it is not detailed in this manual.

[0175] In summary, the containerized deployment solution proposed in this embodiment significantly reduces the complexity of containerized deployment of graph databases, improves resource utilization efficiency and deployment stability, and solves the problems of complex parameters, insufficient resource perception, and difficulty in adapting to multi-node environments in existing technologies.

[0176] Example 2: A containerized deployment tool for a graph database, used to perform adaptive deployment on the current node based on key parameters input by the user, such as Figure 2 Shown, including:

[0177] The matching unit 100 is configured to match the key parameters with preset dynamic templates to obtain corresponding candidate templates, where each dynamic template includes placeholders, assignment rules, and dependency relationships corresponding to each placeholder;

[0178] A generating unit 200 is configured to generate a corresponding dependency graph by using the corresponding placeholders in the obtained candidate templates as points and the dependency relationships as edges, wherein the dependency graph is a directed acyclic graph;

[0179] A derivation unit 300 is configured to derive the actual value corresponding to each placeholder in turn according to the corresponding assignment rules based on the dependency graph to generate corresponding deployment parameters;

[0180] A deployment unit 400 is configured to deploy a container on a current node based on the deployment parameters;

[0181] The dependency graph includes placeholders corresponding to the adaptive memory configuration. The derivation unit 300 simulates the memory load of the current node based on the assignment rule corresponding to the placeholder to obtain the corresponding memory index; and predicts the memory demand based on the memory index to obtain the corresponding prediction result; based on the prediction result, generates an adaptive memory configuration corresponding to the current node and assigns a value to the corresponding placeholder.

[0182] This embodiment is a device embodiment corresponding to embodiment 1. For relevant details, please refer to the partial description of embodiment 1.

[0183] Example 3: A containerized deployment method for a graph database, used in a single-node deployment scenario, includes the following steps.

[0184] Pre-deploy the deployment tool described in Example 2 to the current node;

[0185] Get key parameters input by the user;

[0186] Based on the key parameters, the deployment tool executes the deployment method described in Example 1 to deploy the corresponding graph database to the current node.

[0187] Example 4: A containerized deployment method for a graph database, used in a multi-node deployment scenario, comprising the following steps:

[0188] The central node receives key parameters input by the user;

[0189] The central node sends performance evaluation instructions to each NUMA node, and each NUMA node performs performance evaluation based on preset performance evaluation rules to obtain corresponding performance scores;

[0190] The central node selects several NUMA nodes as target nodes based on the performance scores of each NUMA node and the preset deployment decision rules;

[0191] The central node sends the key parameters and the deployment tool described in Example 2 to each target node;

[0192] The deployment tool executes the deployment method described in Example 1 on the target node based on the key parameters, performs adaptive deployment on the target node, and transmits the corresponding deployment status and logs back to the central node via the SSH standard stream during the deployment process.

[0193] In this embodiment, the placeholder corresponding to cpu_bindings is assigned a value directly based on the CPU logical core set of the target node.

[0194] Example 5: A containerized deployment method for a graph database, used in a multi-node deployment scenario, comprising the following steps:

[0195] The central node receives key parameters input by the user;

[0196] The central node sends the key parameters and the deployment tool described in Example 2 to each NUMA node;

[0197] The deployment tool executes the deployment method described in Example 1 at the target node based on the key parameters, makes a deployment decision at the target node, and cancels the deployment or performs adaptive deployment based on the obtained deployment decision;

[0198] When the deployment decision is to cancel the deployment, ending the derivation of the dependency graph;

[0199] When the deployment decision is to deploy a container, the placeholder corresponding to cpu_bindings is assigned a value based on the CPU logical core set of the current node, and the actual value corresponding to the subsequent placeholder is further derived based on the corresponding dependency graph.

[0200] In this embodiment, the corresponding deployment status and logs during the deployment process are transmitted back to the central node via the SSH standard stream.

[0201] In this embodiment, the dependency graph generated by each deployment tool based on the key parameters includes a placeholder corresponding to cpu_bindings, and the assignment rules corresponding to the placeholder include performance evaluation rules and deployment decision rules, so that deployment decisions are made in the process of deriving the cpu_bindings placeholder based on the dependency graph, and according to the decision result, the placeholder corresponding to cpu_bindings is assigned a value to bind the current node, or the container deployment to the current node is canceled.

[0202] Since generating context objects based on corresponding dynamic templates is a lightweight operation that does not consume node resources, and based on cost and business needs, graph databases are mostly deployed to 3 or 5 nodes. In such scenarios, Example 4 first performs a performance evaluation on each NUMA node. The deployment efficiency of the solution of deploying containers on the target node is not much different from that of the solution of combining performance evaluation and parameter derivation in this embodiment.

[0203] Thanks to the development of container technology, graph databases have successfully solved compatibility issues on multiple operating systems through containerized deployment. However, due to the inherent complexity of graph database cluster systems, performance and functionality must be adjusted and controlled through configuration during cluster deployment. Especially in multi-node cluster deployment scenarios, the same deployment process must be repeated on each node, even in a container environment. Therefore, correctly implementing graph database containerization and achieving rapid deployment has become a key issue.

[0204] In existing technologies, in addition to directly assigning relevant parameters based on default values, it is necessary to manually configure the corresponding deployment parameters for each node where a container is deployed, and send fixed deployment parameters or instructions to the corresponding node for container deployment. In multi-node deployment scenarios, manual parameter configuration is also required for each node that enters the container deployment, which is inefficient and lacks adaptability to the operating environment.

[0205] With reference to the above-mentioned Examples 4 and 5, the present invention uniformly distributes key parameters (the number is far less than the deployment parameters) and deployment tools through a central node, enabling each node to independently detect resources, make intelligent decisions, dynamically derive deployment parameters and independently complete deployment. At the same time, the deployment status is transmitted back in real time through the SSH standard stream, realizing the organic combination of central unified control and node local adaptive deployment, taking into account deployment flexibility and controllability, and significantly improving the efficiency and success rate of large-scale container deployment in heterogeneous environments; it can be widely used in containerized deployment scenarios of large-scale graph database systems, and is particularly suitable for enterprise-level application scenarios in heterogeneous environments and where resources change frequently and dynamically.

[0206] Example 6: A containerized deployment system for a graph database, used to execute the containerized deployment method described in any one of Examples 3 to 5.

[0207] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0208] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0210] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0211] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0212] It should be noted that:

[0213] References in this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, appearances of the phrases "one embodiment" or "an embodiment" in various places throughout this specification do not necessarily refer to the same embodiment.

[0214] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0215] Furthermore, it should be noted that the specific embodiments described in this specification may vary in the shapes and names of their components. Any equivalent or simple variations based on the structure, features, and principles described in the patented concept of this invention are included within the scope of protection of this patent. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments, and these modifications, as long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, shall fall within the scope of protection of this invention.

Claims

1. A containerized deployment method for a graph database, characterized in that: Based on the key parameters entered by the user, adaptive deployment is performed on the current node, including the following steps: Based on the key parameters, matching is performed with preset dynamic templates to obtain corresponding candidate templates, each dynamic template including placeholders, assignment rules and dependency relationships corresponding to each placeholder; The corresponding placeholders in the obtained candidate templates are used as points and the dependency relationships are used as edges to generate a corresponding dependency graph, which is a directed acyclic graph; Based on the dependency graph, the actual value corresponding to each placeholder is deduced in sequence according to the corresponding assignment rules, and the corresponding deployment parameters are generated; Deploy the container on the current node based on the deployment parameters; The dependency graph includes placeholders corresponding to the adaptive memory configuration, and the steps of assigning values ​​to the placeholders corresponding to the adaptive memory configuration based on corresponding assignment rules are as follows: Simulate the memory load of the current node to obtain the corresponding memory indicators; Predicting memory demand based on the memory indicator to obtain corresponding prediction results; Based on the prediction result, an adaptive memory configuration corresponding to the current node is generated.

2. The containerized deployment method of a graph database according to claim 1, characterized in that: When deploying on multiple nodes, the steps for selecting NUMA nodes for container deployment are also included. Specifically: Each NUMA node performs performance evaluation based on preset performance evaluation rules and obtains a corresponding performance score; Select several NUMA nodes as target nodes based on the performance scores of each NUMA node and the preset deployment decision rules; The central node distributes the key parameters input by the user to each target node, and each target node performs adaptive deployment based on the key parameters.

3. The containerized deployment method of a graph database according to claim 1, characterized in that: When performing multi-node deployment, the central node distributes the key parameters entered by the user to each NUMA node, and each NUMA node automatically makes a deployment decision based on the key parameters, and cancels the deployment or performs an adaptive deployment based on the resulting deployment decision; wherein, based on the key parameters, a dependency graph corresponding to the current node is generated, wherein the dependency graph includes a placeholder corresponding to cpu_bindings, and the assignment rules corresponding to the placeholder include performance evaluation rules and deployment decision rules; When assigning values ​​to the placeholders corresponding to cpu_bindings based on the corresponding assignment rules: Performing a performance evaluation on the current node based on the performance evaluation rules to obtain a performance score corresponding to the current node; Generate a deployment decision corresponding to the current node based on the deployment decision rule and the performance score corresponding to each NUMA node; When the deployment decision is to cancel the deployment, ending the derivation of the dependency graph; When the deployment decision is to deploy a container, the placeholder corresponding to cpu_bindings is assigned a value based on the CPU logical core set of the current node, and the actual value corresponding to the subsequent placeholder is further derived based on the corresponding dependency graph.

4. The containerized deployment method of a graph database according to claim 2 or 3, characterized in that: The steps of performing performance evaluation on the current node based on the performance evaluation rule include: Perform load simulation on the CPU of the current node to obtain performance indicators corresponding to the current node, including memory bandwidth, memory access latency, CPU soft interrupt processing time ratio, and CPU idle ratio; Each performance indicator is normalized and then weighted and summed to obtain a corresponding performance score. A higher performance score indicates better performance of the corresponding NUMA node.

5. The containerized deployment method for a graph database according to claim 2 or 3, characterized in that: After receiving the key parameters input by the user, the current node deploys the container based on the key parameters, and transmits the corresponding deployment status and logs back to the central node via the SSH standard stream during the deployment process.

6. The containerized deployment method for a graph database according to any one of claims 1 to 3, characterized in that: The adaptive memory configuration is memory_limit. The steps to assign a value to the placeholder corresponding to memory_limit based on the corresponding assignment rules are as follows: Perform a load simulation on the memory of the current node to obtain corresponding memory indicators, including peak heap memory usage and cache hit rate; Predicting the memory demand corresponding to the current node based on the memory indicator using a preset linear regression model to obtain a first prediction result; Predicting the memory demand corresponding to the current node based on the memory indicator using a preset random forest regression model to obtain a second prediction result; Performing weighted fusion on the first prediction result and the second prediction result to obtain a target memory requirement range; A corresponding maximum memory limit value is calculated based on the target memory requirement range, and a value is assigned to the placeholder corresponding to memory_limit.

7. The containerized deployment method for a graph database according to any one of claims 1 to 3, characterized in that: The key parameters include service type parameters and / or environment configuration key parameters.

8. The containerized deployment method for a graph database according to claim 7, characterized in that: Dynamic templates include: A service template corresponding to the service type parameter, the service template being used to indicate a corresponding management backend interface; a policy template corresponding to the key environment configuration parameter, the policy template being used to indicate a mounting policy or a port mapping policy; There are one or more candidate templates that match the key parameters input by the user, and a corresponding dependency graph is generated based on all matching candidate templates.

9. A containerized deployment tool for a graph database, characterized in that: Used to perform adaptive deployment on the current node based on key parameters entered by the user, including: A matching unit, configured to match the key parameters with preset dynamic templates to obtain corresponding candidate templates, each dynamic template including placeholders, assignment rules and dependency relationships corresponding to each placeholder; A generating unit, configured to use the corresponding placeholders in the obtained candidate templates as points and the dependency relationships as edges to generate a corresponding dependency graph, wherein the dependency graph is a directed acyclic graph; A derivation unit, configured to derive the actual value corresponding to each placeholder in turn according to the corresponding assignment rules based on the dependency graph, and generate corresponding deployment parameters; A deployment unit, configured to deploy a container on a current node based on the deployment parameters; The dependency graph includes placeholders corresponding to the adaptive memory configuration. The derivation unit simulates the memory load of the current node based on the assignment rules corresponding to the placeholder to obtain corresponding memory indicators; and predicts memory requirements based on the memory indicators to obtain corresponding prediction results; based on the prediction results, generates an adaptive memory configuration corresponding to the current node and assigns values ​​to the corresponding placeholders.

10. A containerized deployment system for a graph database, characterized in that: Used to execute the containerized deployment method according to any one of claims 1 to 8.

Images