Data receiving and managing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data management method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a management demand required by a resource pool; determining a service logic for a management demand, and writing a newly added service module file based on the service logic; obtaining a resource pool network planning table, and according to the resource pool network planning table, determining resource attribute information and configuration information needing customized management; writing a YAML configuration file for the target business scene according to the resource attribute information and the configuration information; and the data of the resource pool is managed based on the service module file and the YAML configuration file, so that the decoupling of the management logic and the configuration parameters is realized, a user can quickly and efficiently complete the automatic management of different resource pools only by replacing the customized YAML configuration file without modifying or maintaining a large number of script codes, and the user experience is improved. Therefore, the time consumption of pre-resource management is greatly shortened, and the complexity of operation and maintenance management is reduced.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and in particular to a data management method, a data management device, an electronic device, and a readable storage medium. Background Technology

[0002] Currently, after the network components of various cloud computing resource pools are deployed, it is usually necessary to automate the management of front-end resources (such as clusters, network elements, various specifications, front-end switches, etc.) to ensure that all functions of the resource pool environment are normal and available. Since manually executing the creation process is very time-consuming, existing technologies usually adopt the method of writing a single automated script to call the interface of the control plane component to perform resource creation and management operations in batches.

[0003] However, this single-script-based automation solution has significant drawbacks: There are numerous cloud computing resource pools, and each pool's actual situation and specific management needs differ. For example, the number of clusters, configuration parameters, or required specifications may vary depending on the size of the resource pool and network planning. Therefore, existing solutions require operations and maintenance personnel or developers to specifically manage, modify, and maintain the corresponding automation scripts for each different resource pool. This frequent script modification workflow leads to high time costs and complex operations and maintenance burdens, limiting the efficiency and flexibility of rapid deployment of large-scale resource pools. Summary of the Invention

[0004] The present invention provides a data management method, apparatus, electronic device, and readable storage medium to overcome or at least partially solve the above-mentioned problems.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a data management method, including: Determine the management requirements for the resource pool; Determine the business logic for the management requirements, and write the new business module file based on the business logic; Obtain the resource pool network planning table, and based on the resource pool network planning table, determine the resource attribute information and configuration information that need to be customized for management; Based on the resource attribute information and the configuration information, write a YAML configuration file for the target business scenario; The data in the resource pool is managed based on the business module file and the YAML configuration file.

[0006] Optionally, before the step of determining the business logic for the management requirements and writing the new business module file based on the business logic, the method further includes: Determine whether a similar business module file related to the management requirement already exists in the target business scenario; If it is determined that there is no such business module file, then the steps of determining the business logic for the management requirements and writing the new business module file based on the business logic are executed.

[0007] Optionally, the step of performing data management operations on the resource pool based on the business module file and the YAML configuration file includes: The management program used to execute the business logic and the business module file are uploaded to the execution node device of the resource pool; The execution node device is used to obtain the instructions entered by the user, and based on the instructions, a managed mode command line for controlling the managed program to execute the business logic is determined, as well as the target YAML configuration file corresponding to the managed mode command line; The format of the target YAML configuration file is validated, and when the validation result meets the preset conditions, the target YAML configuration file is loaded to obtain the resource configuration information of the resource pool for the target business scenario. Read the resource configuration information, convert the resource configuration information into a resource configuration combination, and temporarily store the resource configuration combination in the memory device of the execution node device; According to the command line of the management mode, the business module file is executed, and when the business module file is executed, the resource configuration combination is read to perform management operation on the data of the resource pool.

[0008] Optionally, before the step of executing the business module file according to the management mode command line, the method further includes: The log module is initialized based on the resource configuration combination temporarily stored in the memory device.

[0009] Optionally, it also includes: Generate the management execution result for the data to be managed; Log information containing the management execution results and the resource configuration combination is written to the log module.

[0010] Optionally, it also includes: If an error is detected in the execution result of the management process, the execution of the business module file is stopped, and an error log is generated. The error log is entered into the log module.

[0011] Optionally, it also includes: The error log is used to determine the created resources that need to be rolled back; Based on the rollback scenario command entered by the user, a reverse business scenario is triggered, which is used to delete and roll back the created resources.

[0012] Secondly, embodiments of this application provide a data management device, including: The management requirement determination module is used to determine the management requirements of the resource pool; The business module file writing module is used to determine the business logic for the management requirements and to write new business module files based on the business logic. The resource attribute information and configuration information acquisition module is used to acquire the resource pool network planning table and determine the resource attribute information and configuration information that need to be customized and managed based on the resource pool network planning table. The YAML configuration file writing module is used to write YAML configuration files for the target business scenario based on the resource attribute information and the configuration information. The management operation execution module is used to perform management operations on the data of the resource pool based on the business module file and the YAML configuration file.

[0013] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0014] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0015] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0016] The embodiments of the present invention have the following advantages: This invention, through the following embodiments, decouples the management logic from the configuration parameters. It determines the management requirements of a resource pool; identifies the business logic for these requirements and writes new business module files based on this logic; obtains a resource pool network planning table and, based on this table, determines the resource attribute information and configuration information that need to be customized for management; writes a YAML configuration file for the target business scenario based on the resource attribute information and the configuration information; and performs management operations on the data of the resource pool based on the business module files and the YAML configuration file. This decouples the management logic from the configuration parameters, allowing users to quickly and efficiently complete automated management of different resource pools without modifying or maintaining a large amount of script code, simply by replacing the customized YAML configuration file. This significantly reduces the time required for front-end resource management and lowers the complexity of operation and maintenance management. Attached Figure Description

[0017] Figure 1 This is a flowchart of the steps of a data management method provided in an embodiment of the present invention; Figure 2 This is a system module execution flowchart provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating a data management method provided in an embodiment of the present invention; Figure 4 This is a structural block diagram of a data management device provided in an embodiment of the present invention; Figure 5 This is a hardware structure block diagram of an electronic device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] Reference Figure 1 The diagram illustrates a flowchart of a data management method provided in an embodiment of the present invention, which may specifically include the following steps: Step 101: Determine the management requirements for the resource pool; Step 102: Determine the business logic for the management requirements, and write the new business module file based on the business logic; Step 103: Obtain the resource pool network planning table, and determine the resource attribute information and configuration information that need to be customized and managed based on the resource pool network planning table; Step 104: Based on the resource attribute information and the configuration information, write a YAML configuration file for the target business scenario; Step 105: Perform data management operations on the resource pool based on the business module file and the YAML configuration file.

[0022] In this embodiment of the invention, the management requirements of the resource pool can be determined to clarify the specific business objectives and scope of this automated operation (e.g., whether to create a cluster or manage network elements).

[0023] Resource pool: The physical or virtual environment in which various cloud computing components and virtual network resources are deployed, usually referring to a data center or a group of clusters.

[0024] Management requirements: These refer to the specific task requirements for the creation, configuration, or deployment of pre-resources (such as clusters, network elements, specifications, etc.) that must be achieved through automated programs after the deployment or changes of the resource pool.

[0025] In this embodiment of the invention, the business logic for the management requirements can be determined, and new business module files can be written based on the business logic to separate the execution flow (business logic) of the management operation from the specific configuration parameters, and code reusability can be achieved by writing general code.

[0026] Business logic refers to the fixed and general sequence of operations and decision-making processes required to complete a specific management task, such as "first create the cluster, then manage the network element".

[0027] Business module file: A general code file (usually a script) containing the core business logic of the management program. It defines the management and rollback process by combining calls to the SDK methods of the management plane components.

[0028] In this embodiment of the invention, a resource pool network planning table can be accurately obtained, and based on the resource pool network planning table, the resource attribute information and configuration information that need to be customized for management can be determined. Based on the actual planning and scale of the resource pool, all specific parameters and resource types required for customized management operations can be collected and determined as the data source for program execution.

[0029] Source pool network planning table: An external document or specification that defines in detail the topology, IP range, number, and specifications required for various resources in the current resource pool.

[0030] Resource attribute information: refers to the type of resource that needs to be created or configured (e.g., cluster, network element, SLA specification).

[0031] Configuration information: refers to the specific values ​​or parameters corresponding to the resource attributes (e.g., the number of clusters is 3, the IP address range is 10.1.xx, and the specification value is 99.99%).

[0032] In this embodiment of the invention, YAML configuration files can be written for the target business scenario based on the resource attribute information and the configuration information, so as to solidify the customized parameters collected in the previous step into a data input file in a YAML format that can be recognized and verified by the program, thereby achieving complete decoupling of code and data.

[0033] Target business scenario: refers to the specific application background of this management operation, such as "initial deployment of the East China resource pool" or "expansion and upgrade".

[0034] YAML configuration file: A structured data serialization format file that carries all customized resource attribute information and configuration information, used to guide parameter filling during program execution.

[0035] In this embodiment of the invention, data management operations can be performed on the resource pool based on the business module file and the YAML configuration file to start an automated program, load and use customized parameters in the YAML configuration file, drive the logical flow defined in the business module file, and complete automated management operations such as resource creation and configuration.

[0036] Data in the resource pool refers to various virtual network front-end resources such as clusters, network elements, specifications, and switches that are created, configured, updated, or deleted under the management program.

[0037] Managed operations: The automated program performs management actions such as creating, configuring, and deleting resource pool data by calling the SDK.

[0038] This invention, through the following embodiments, decouples the management logic from the configuration parameters. It determines the management requirements of a resource pool; identifies the business logic for these requirements and writes new business module files based on this logic; obtains a resource pool network planning table and, based on this table, determines the resource attribute information and configuration information that need to be customized for management; writes a YAML configuration file for the target business scenario based on the resource attribute information and the configuration information; and performs management operations on the data of the resource pool based on the business module files and the YAML configuration file. This decouples the management logic from the configuration parameters, allowing users to quickly and efficiently complete automated management of different resource pools without modifying or maintaining a large amount of script code, simply by replacing the customized YAML configuration file. This significantly reduces the time required for front-end resource management and lowers the complexity of operation and maintenance management.

[0039] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0040] In an optional embodiment of the present invention, before the steps of determining the business logic for the management requirement and writing the new business module file based on the business logic, it is determined whether there is already a similar business module file in the target business scenario that is related to the management requirement. This is to check whether there is a reusable business module file before starting new code writing work, so as to avoid duplication of work and improve efficiency.

[0041] Target business scenario: refers to the specific application background or type of this management operation (e.g., cluster initialization, network element expansion).

[0042] Similar business module files: refers to existing business module files that are applicable to the same type of business logic and can be reused.

[0043] In this embodiment of the invention, when it is determined that there is no such similar business module file, the steps of determining the business logic for the management requirements and writing a new business module file based on the business logic can be executed. After confirming that there is no reusable code, the user can write targeted code to separate the execution flow (business logic) of the management operation from the configuration parameters, laying the foundation for automated execution.

[0044] In this embodiment of the invention, by adding a reuse judgment mechanism for business module files and achieving complete decoupling between management logic (business module files) and configuration parameters (YAML configuration files), users no longer need to frequently modify and maintain a large number of automated scripts for each resource pool. They can quickly and efficiently complete the automated management of different resource pools simply by modifying customized YAML configuration files, which greatly shortens the time spent on front-end resource management and reduces the complexity of operation and maintenance management.

[0045] In an optional embodiment of the present invention, the step of performing data management operations on the resource pool based on the business module file and the YAML configuration file includes: The management program used to execute the business logic and the business module file are uploaded to the execution node device of the resource pool; The execution node device is used to obtain the instructions entered by the user, and based on the instructions, a managed mode command line for controlling the managed program to execute the business logic is determined, as well as the target YAML configuration file corresponding to the managed mode command line; The format of the target YAML configuration file is validated, and when the validation result meets the preset conditions, the target YAML configuration file is loaded to obtain the resource configuration information of the resource pool for the target business scenario. Read the resource configuration information, convert the resource configuration information into a resource configuration combination, and temporarily store the resource configuration combination in the memory device of the execution node device; According to the command line of the management mode, the business module file is executed, and when the business module file is executed, the resource configuration combination is read to perform management operation on the data of the resource pool.

[0046] In this embodiment of the invention, the management program for executing the business logic and the business module file can be uploaded to the execution node device of the resource pool to complete the environment preparation work before automated execution and ensure that all necessary program code (main program and business logic) exists in the runtime environment.

[0047] Management Program: The main program that is executed automatically, including control modules, configuration modules, etc., and is responsible for receiving instructions, scheduling and coordinating the operation of the entire management process.

[0048] Business module files: These are code files containing the specific business logic (SDK calls) for the management operation and are key components of the management program.

[0049] Execution node device: The server, virtual machine, or container that actually runs the managed program and has the ability to call the management plane interface.

[0050] In this embodiment of the invention, the execution node device can also acquire instructions entered by the user, determine the managed mode command line for controlling the managed program to execute the business logic based on the instructions, and the target YAML configuration file corresponding to the managed mode command line, so as to receive external input from the user and identify the execution mode and customized parameter file of this operation. This is a key step in starting the automated process.

[0051] Command line for managed mode: Commands entered by the user in the terminal to specify the mode of this managed operation (e.g., initialization, expansion, etc.) and the target configuration file path.

[0052] Target YAML configuration file: The YAML file specified in the command line that contains all the customized parameters for this management operation.

[0053] In this embodiment of the invention, the format of the target YAML configuration file can also be validated, and when the validation result meets the preset conditions, the target YAML configuration file is loaded to obtain the resource configuration information of the resource pool for the target business scenario, so as to ensure the validity and integrity of the input data, prevent program execution failure due to format errors or missing parameters, and obtain the original configuration data.

[0054] The validation process can be performed by a management program to check whether the syntax and structure of the YAML configuration file conform to the predetermined specifications.

[0055] Resource configuration information: A raw data set containing all customized resource attributes and configuration information, obtained after loading from the YAML configuration file.

[0056] In this embodiment of the invention, the resource configuration information can be read, the resource configuration information can be converted into a resource configuration combination, and the resource configuration combination can be temporarily stored in the memory device of the execution node device. This transforms the original configuration data into a unified and efficient data structure (resource configuration combination) for internal processing and stores it in memory for subsequent business modules to call quickly.

[0057] Resource configuration combination: After the resource configuration information is read, parsed and transformed by the program, it is stored in memory in a structured data format that can be directly used for business logic execution.

[0058] Memory device: The memory of the execution node device where the managed program runs, used for temporary and fast data storage and access.

[0059] In this embodiment of the invention, the business module file can be executed according to the management mode command line, and the resource configuration combination can be read during the execution of the business module file to perform management operations on the data of the resource pool.

[0060] In the actual implementation, the management program starts the business module, which reads the customized parameters in memory and calls the SDK to complete the creation and configuration of resources.

[0061] Executing the business module file: The process by which the management program starts and runs the corresponding business logic code according to the command line.

[0062] Performing management operations: The business module calls the underlying SDK interface to actually create, modify, or delete resources in the resource pool.

[0063] The embodiments of the present invention constitute an efficient, configuration-driven automated execution system, which enables users to quickly and efficiently complete the automated management of different resource pools without modifying the script code for each resource pool, simply by replacing the customized YAML configuration file. This significantly reduces the time required for front-end resource management and lowers the complexity of operation and maintenance management.

[0064] In an optional embodiment of the present invention, before the step of executing the business module file according to the management mode command line, the log module can be initialized according to the resource configuration combination temporarily stored in the memory device to ensure that the log system is in a ready state before the management operation begins. Since log recording may require recording configuration parameters (such as resource name, quantity, etc.) or specific log levels, this step uses the loaded customized configuration (i.e., the resource configuration combination in memory) to parameterize and start the log module.

[0065] The resource configuration combination temporarily stored in the memory device refers to all customized parameters and data structures that the management program reads, verifies, and transforms from the YAML configuration file and stores in the running memory for quick access by the program.

[0066] Initialization: Configure and set the logging module, including determining the log level, output format, storage path, etc., to prepare it to receive and process log information generated during subsequent management operations.

[0067] Log Module: A component in a managed program, specifically designed to record program running status, execution results, error messages, and operation details.

[0068] In this embodiment of the invention, by completely decoupling the management logic (business module file) from the configuration parameters (YAML configuration file), users no longer need to frequently modify and maintain a large number of automated scripts for each resource pool. They can quickly and efficiently complete the automated management of different resource pools simply by modifying the customized YAML configuration file, which greatly shortens the time spent on front-end resource management and reduces the complexity of operation and maintenance management.

[0069] In an optional embodiment of the present invention, it further includes: Generate the management execution result for the data to be managed; Log information containing the management execution results and the resource configuration combination is written to the log module.

[0070] In this embodiment of the invention, a management execution result can be generated for the data to be managed, so as to record the final state of the business module file after the management operation is performed, and determine whether the automated deployment or change is successful.

[0071] Management execution result: refers to the final status feedback generated after the business module file calls the underlying SDK interface to perform management operation on the resource pool data, which is usually a success or failure indication.

[0072] In this embodiment of the invention, log information including the management execution result and the resource configuration combination can be written into the log module to permanently and traceably record the final state of this management operation and all customized input parameters (configuration data) in the log system, which facilitates subsequent auditing and error investigation.

[0073] Resource configuration combination: The collection of all customized parameters that the management program reads from the YAML configuration file, transforms, and temporarily stores in memory.

[0074] Log information: Records structured or unstructured data of program running status, execution process details, error reasons, and the combination of managed execution results and resource configurations used in the execution.

[0075] Log module: The component in the management program responsible for receiving, formatting, storing and managing all execution logs.

[0076] This invention achieves complete decoupling between management logic (business module files) and configuration parameters (YAML configuration files), enabling users to quickly and efficiently complete automated management of different resource pools without frequently modifying and maintaining a large number of automated scripts for each resource pool. This significantly reduces the time required for front-end resource management and lowers the complexity of operation and maintenance.

[0077] In an optional embodiment of the present invention, it further includes: If an error is detected in the execution result of the management process, the execution of the business module file is stopped, and an error log is generated. The error log is entered into the log module.

[0078] In this embodiment of the invention, when an error is determined to exist in the management execution result, the execution of the business module file can be stopped and an error log can be generated to ensure that the management operation can stop immediately when it encounters a fatal error, prevent resource chaos or waste that may be caused by further execution, and capture specific information about the error in a timely manner.

[0079] Management execution result: The status feedback received by the program after the business module file calls the underlying interface to perform the operation, specifically referring to the failure or exception status.

[0080] The execution of the business module file is suspended: After the management program detects an error, it immediately stops the execution of the remaining code in the business module file and exits the current automation process.

[0081] Error logs: Data records generated for failed or abnormal execution of management tasks, including key information such as error type, timestamp, reason for failure, and execution context.

[0082] In this embodiment of the invention, the error log can be entered into the log module to persistently store the generated error information, ensuring that the fault information is not lost, and providing a reliable basis for subsequent maintenance personnel to conduct fault investigation, problem location and rollback decisions.

[0083] Error Log: This is the data record containing fault information generated in the previous step.

[0084] Log Module: The component in the managed program responsible for collecting, formatting, storing and managing all execution records and error information, and is the core of system traceability.

[0085] This invention, through a complete decoupling of management logic (business module files) and configuration parameters (YAML configuration files), eliminates the need for users to frequently modify and maintain numerous automated scripts for each resource pool. Instead, users can quickly and efficiently automate the management of different resource pools simply by modifying customized YAML configuration files, significantly reducing the time required for front-end resource management and lowering the complexity of operation and maintenance.

[0086] In an optional embodiment of the present invention, it further includes: The error log is used to determine the created resources that need to be rolled back; Based on the rollback scenario command entered by the user, a reverse business scenario is triggered, which is used to delete and roll back the created resources.

[0087] In this embodiment of the invention, the created resources that need to be rolled back can be determined based on the error log. After the management operation fails, the execution information recorded in the error log can be used to accurately locate which resources were successfully created or partially configured before the failure occurred, thereby defining the scope of cleanup and recovery.

[0088] Error log: Contains detailed records of failed or abnormal management operations, used to trace the cause of the failure and determine the successful operation.

[0089] Created resources: These are resources that have been successfully created by the program through the interface call before the management process fails, but have not yet reached their final target state.

[0090] In this embodiment of the invention, a reverse business scenario can be triggered based on a rollback scenario command entered by the user. The reverse business scenario is used to delete and roll back the created resources. After determining the rollback scope, the user initiates a command, and the system automatically executes the pre-written cleanup logic to restore the resource pool to a safe and clean initial state.

[0091] Rollback scenario command: The user enters a command in the terminal that explicitly requests the system to initiate resource cleanup and recovery operations.

[0092] Reverse business scenario: Pre-write the cleanup process in the business module file that corresponds to the forward management logic. For example, for the operation of creating a cluster, the reverse business scenario is to delete the cluster.

[0093] Delete and rollback: In case of failure during automated deployment, the reverse business scenario is executed to safely and thoroughly delete all created resources and restore the environment to its state before the operation.

[0094] In this embodiment of the invention, by completely decoupling the management logic (business module file) from the configuration parameters (YAML configuration file), users can quickly and efficiently complete the automated management of different resource pools without having to frequently modify and maintain a large number of automated scripts for each resource pool. This significantly reduces the time required for front-end resource management and lowers the complexity of operation and maintenance management.

[0095] For example, data management can be performed in the following manner.

[0096] Assume the management requirement is to automate the creation of necessary front-end resources on a newly deployed East China resource pool (Target DC) to ensure the proper functionality and availability of all virtual network components. This is the business scenario for this operation.

[0097] First, for this "pre-resource initialization" business scenario, determine whether the corresponding business module file already exists.

[0098] Assuming this is a completely new managed application scenario, a new business module file (a Python or Go code file) needs to be written. The core content of this file defines the managed application's business logic: it sequentially calls SDK module methods, writing code to implement a fixed automated process such as "creating a cluster → managing network elements into the cluster → creating SLA specifications." This file defines "what to do."

[0099] Based on the network planning table of the resource pool, determine the parameters that need to be customized for this management project.

[0100] Resource attribute information determines the type that needs to be managed, such as: Cluster, NetElement, Specification.

[0101] The configuration information specifies the concrete values ​​for these types, such as: the number of clusters is 3; the cluster IP range is 10.1.xx; the required SLA specification is named SLA-High, and its value is 99.99%. Execution entity A organizes these customized resource attribute and configuration information and compiles them into a YAML configuration file (e.g., dc_huadong_config.yaml). This file defines **"what parameters to use"**.

[0102] Upload the management program, business module files, and YAML configuration file to the target node, and have the user enter the command line for the management mode required for this execution, for example: . / onboard_cli --mode initial_setup --configdc_huadong_config.yaml.

[0103] At this point, the managed mode command line and the target YAML configuration file are sent to the managed program / control module of the execution node of the resource pool. For example, the input information is the managed mode command line (e.g., --mode initial_setup) typed by the user and the path to the specified target YAML configuration file.

[0104] After receiving the command line and configuration file, the control module first instructs the configuration module to load and verify the YAML configuration file. Upon successful verification, the configuration module reads, converts, and temporarily stores the resource configuration information from the file, such as cluster IPs, quantities, and specifications, in memory, forming a resource configuration combination. Subsequently, the control module matches the corresponding business module file based on the command line and begins execution. The business module reads the resource configuration combination from memory and, based on the parameter "cluster quantity is 3," calls the SDK module to perform the "create 3 clusters" management operation, ultimately completing the customized automated deployment.

[0105] To enable those skilled in the art to better understand the embodiments of the present invention, an example is used below to illustrate the embodiments of the present invention.

[0106] In practical applications, after the deployment of various cloud computing resource pool network components, it is necessary to manage the preceding resources, such as clusters, network elements, various specifications, and various front-end switches, in order for all functions of the resource pool environment to be normal and available. Manually performing the creation is too time-consuming, so the existing method usually involves writing a single script to call the component interface to perform the creation.

[0107] However, this approach still has a problem: the number of resource pools is large, and the actual situation and management requirements of each resource pool are different. For example, the number and configuration of clusters vary depending on the size of the resource pool and the network planning. Therefore, it is necessary to manage and modify the scripts specifically for each resource pool, which still results in a relatively high time cost.

[0108] As mentioned above, users need a more efficient and customizable management method to further reduce the time spent on each front-end resource management process, which is the purpose of this technology.

[0109] Use cases for configuration file-based management plane methods include managing required front-end resources after the deployment of various virtual network management components. Simply modify the configuration file according to the network plan of each resource pool, and the corresponding management scenario can be executed with a single command. (Reference) Figure 2 , Figure 2 This is a system module execution flowchart provided in an embodiment of the present invention. The management method includes an SD module, a business module, a configuration module, a log module, and a control module.

[0110] The main process steps of the technical solution are as follows: Figure 3 As shown, Figure 3This is a flowchart illustrating a data management method provided in this embodiment of the invention. The step "configuration module writing" is the main technical point of this invention. By writing a configuration file, users can quickly and automatically complete the data management scenario simply by filling in the required resource configuration and entering the corresponding commands.

[0111] This technology can optimize the repetitive time-consuming management of most similar virtual network resources through the above usage process.

[0112] The SDK module is used to combine and call the SDKs provided by the corresponding control plane components, and after appropriate rearrangement, provide services to the business modules.

[0113] The configuration module loads the configuration file during the program initialization phase. After verifying whether the format of the configuration file meets expectations, it reads and converts the user-defined resource configuration, and temporarily stores it in memory for the business module to read and execute as needed.

[0114] The business module combines the methods provided by the SDK module according to the needs of a certain type of resource pool or business scenario. Each business scenario corresponds to a command-line mode. After interacting with the user, the business module reads the parameters in memory and executes the corresponding management operations.

[0115] The log module records relevant information during the execution of the configuration module and business module, making it easy to find clues when troubleshooting.

[0116] The control module is responsible for receiving parameters, calling each module in sequence, and displaying the results.

[0117] Specific use cases are as follows: Before deploying or changing the resource pool, first determine the management requirements and confirm whether similar business module files already exist in the business scenario. If so, they can be used directly, proceeding to the next step of writing the configuration module; otherwise, a business module file needs to be written specifically for this business scenario. The writing of this general file should not only be applicable to this management but also adaptable to other similar business scenarios as much as possible. In addition to common parameters such as log parameters and SDK parameters, the configuration file in the configuration module should include the resource attributes and configurations that need to be customized for this management. Enter the command line corresponding to the management mode for this business module in the corresponding resource pool, and the control module will begin executing automated management.

[0118] The execution order of the management procedure is as follows: The first step is for the configuration module to load the configuration file, verify its format, read and convert the user-defined resource configuration, and temporarily store it in memory. The second step is for the logging module to initialize based on the configuration information from the first step. The third step is for the control module to execute the corresponding business module file according to the command line. This module reads the resource configuration combination from memory, calls the SDK module for management, and writes the call logs to the logging module. If an error is encountered, execution will be stopped, an error will be thrown, and the reason for the error will also be recorded in the logging module for easy troubleshooting.

[0119] Here is a specific example: This management method is applied to resource management scenarios after the deployment of virtual network management plane components. Using this method, the necessary pre-requisite resources for the resource pool are automatically created based on the cluster, network element, specification, and various switches configured in the configuration file.

[0120] The first step is to write a business module before deploying or changing the resource pool. This module calls the SDK provided by the management plane component to sequentially create clusters, network elements, add managed network elements to the cluster, launch network elements, create specifications, create switches and turn them on, etc. After each step, the logging module is called to print the corresponding execution logs. If management fails, the logs can be used for troubleshooting. Additionally, a reverse business scenario for deleting specifications, network elements, and clusters needs to be written as a rollback scenario. If errors are found after management is completed or re-management is required, this rollback scenario allows for one-click automated rollback, saving significant time compared to manual cleanup.

[0121] The second step is to determine the number of clusters to be managed and their configuration information such as IP addresses, network element IP addresses, types, hostnames, and various SLA configurations such as cps, qps, and maximum number of connections, based on the network planning table. Then, for the business module scenario developed in the first step, write it into the YAML configuration file according to the format specifications.

[0122] The third step is to log in to the node that can call the management interface component, and upload the management program and the yaml configuration file written in the second step.

[0123] The fourth step is to execute the management procedure. Specify the configuration file and enter the desired mode (corresponding to the business module written in the first step). The corresponding management operation will then be executed automatically. Finally, observe the program output. If the output is successful, the resource management is complete; if it fails, observe the error logs generated by the logging module and investigate. If necessary, you can re-enter the rollback scenario command written in the first step to delete and roll back the created resources, allowing you to re-implement the management process after troubleshooting.

[0124] Furthermore, this solution can be expanded, for example, for use in online timed smoke test inspections: As described in the first step of the above specific embodiment, the smoke test business module is first written, calling the interfaces provided by the SDK module. Taking load balancing as an example, resources such as backend host groups, backend instances, load balancer instances, forwarding policies, certificates, and listeners are created sequentially, and the corresponding execution logs are printed to the log module. Then, the resources created above are deleted in reverse order, leaving no test resources online. The second step is to write the attribute values ​​of the corresponding smoke test resources in the configuration file. The third step is to start the corresponding coroutine in the control module to execute the smoke test business scenario on a regular basis. This embodiment can simulate the operation of an online user, performing timed tests on the entire process from creation to deletion, thereby ensuring the normal operation of basic business. If an anomaly occurs, it can be investigated based on the information provided by the log module, so as to avoid affecting a large number of real users without the administrator's knowledge.

[0125] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0126] Reference Figure 4 The diagram illustrates a structural block diagram of a data management device provided in an embodiment of the present invention, which may specifically include the following modules: The management requirement determination module 401 is used to determine the management requirements of the resource pool. The business module file writing module 402 is used to determine the business logic for the management requirements and to write new business module files based on the business logic. The resource attribute information and configuration information acquisition module 403 is used to acquire the resource pool network planning table and determine the resource attribute information and configuration information that need to be customized and managed based on the resource pool network planning table. YAML configuration file writing module 404 is used to write YAML configuration files for the target business scenario based on the resource attribute information and the configuration information. The management operation execution module 405 is used to perform management operations on the data of the resource pool based on the business module file and the YAML configuration file.

[0127] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0128] In addition, embodiments of the present invention also provide an electronic device, such as... Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503, and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504. Memory 503 is used to store computer programs; When processor 501 executes a program stored in memory 503, it implements any of the data management methods described in the above embodiments: The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0129] The communication interface is used for communication between the aforementioned terminal and other devices.

[0130] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0131] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0132] like Figure 6As shown, in another embodiment of the present invention, a computer-readable storage medium 601 is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the data management method described in the above embodiments.

[0133] This application also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described data management method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0134] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0135] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0137] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A data management method, characterized in that, include: Determine the management requirements for the resource pool; Determine the business logic for the management requirements, and write the new business module file based on the business logic; Obtain the resource pool network planning table, and based on the resource pool network planning table, determine the resource attribute information and configuration information that need to be customized for management; Based on the resource attribute information and the configuration information, write a YAML configuration file for the target business scenario; The data in the resource pool is managed based on the business module file and the YAML configuration file.

2. The method according to claim 1, characterized in that, Before the steps of determining the business logic for the management requirements and writing the new business module file based on the business logic, the method further includes: Determine whether a similar business module file related to the management requirement already exists in the target business scenario; If it is determined that there is no such business module file, then the steps of determining the business logic for the management requirements and writing the new business module file based on the business logic are executed.

3. The method according to claim 1, characterized in that, The steps of performing data management operations on the resource pool based on the business module file and the YAML configuration file include: The management program used to execute the business logic and the business module file are uploaded to the execution node device of the resource pool; The execution node device is used to obtain the instructions entered by the user, and based on the instructions, a managed mode command line for controlling the managed program to execute the business logic is determined, as well as the target YAML configuration file corresponding to the managed mode command line; The format of the target YAML configuration file is validated, and when the validation result meets the preset conditions, the target YAML configuration file is loaded to obtain the resource configuration information of the resource pool for the target business scenario. Read the resource configuration information, convert the resource configuration information into a resource configuration combination, and temporarily store the resource configuration combination in the memory device of the execution node device; According to the command line of the management mode, the business module file is executed, and when the business module file is executed, the resource configuration combination is read to perform management operation on the data of the resource pool.

4. The method according to claim 3, characterized in that, Before the step of executing the business module file according to the management mode command line, the method further includes: The log module is initialized based on the resource configuration combination temporarily stored in the memory device.

5. The method according to claim 4, characterized in that, Also includes: Generate the management execution result for the data to be managed; Log information containing the management execution results and the resource configuration combination is written to the log module.

6. The method according to claim 5, characterized in that, Also includes: If an error is detected in the execution result of the management process, the execution of the business module file is stopped, and an error log is generated. The error log is entered into the log module.

7. The method according to claim 6, characterized in that, Also includes: The error log is used to determine the created resources that need to be rolled back; Based on the rollback scenario command entered by the user, a reverse business scenario is triggered, which is used to delete and roll back the created resources.

8. A data management device, characterized in that, include: The management requirement determination module is used to determine the management requirements of the resource pool; The business module file writing module is used to determine the business logic for the management requirements and to write new business module files based on the business logic. The resource attribute information and configuration information acquisition module is used to acquire the resource pool network planning table and determine the resource attribute information and configuration information that need to be customized and managed based on the resource pool network planning table. The YAML configuration file writing module is used to write YAML configuration files for the target business scenario based on the resource attribute information and the configuration information. The management operation execution module is used to perform management operations on the data of the resource pool based on the business module file and the YAML configuration file.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method as described in claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the method as described in claims 1-7.

Images