Container system process life cycle management method and system

Abstract

The invention discloses a container system process life cycle management method and system, and relates to the technical field of computers, and the method comprises the following steps: constructing a container cluster system covering a single machine and a cluster mode; a configuration management mapping scheme from container process management to host process management is designed, and process configuration in a container is converted into process configuration on a host by using a namespace and a resource management subsystem of a Linux kernel. According to the container system process life cycle management method and system, the resource utilization rate and the system performance are improved through fine resource allocation and NUMA optimization, under the NUMA architecture, by identifying and configuring NUMA nodes, it is ensured that the process preferentially uses a memory on a specific NUMA node, memory access delay is reduced, the data processing efficiency is improved, and the service life of the container system process is prolonged. CPU and memory resource allocation is dynamically adjusted according to the priority and the dependency relationship of the processes, and it is ensured that the key processes obtain enough resources to maintain the optimal performance.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for managing the lifecycle of a container system process. Background Art

[0002] With the popularization of cloud computing technology, the demand for cloud-based business is growing. As one of the core technologies for modern application deployment and management, containers are known for their lightweight, high efficiency and portability. Compared with virtual machine container technology, they have more outstanding performance advantages. Therefore, traditional business containerization transformation is often more popular with users than other cloud solutions. Due to historical reasons, business complexity and other factors, traditional business system design is generally more complex. It is a multi-process, multi-task system. In physical host and virtual machine environments, solutions such as supervisor and systemd are often used for process management.

[0003] However, in the context of containers, such business system transformation faces new challenges. The design concept of "one container, one process" originates from the early best practices of containerization technology, especially from the official design philosophy of Docker and the single responsibility principle of UNIX / Linux. It is also driven by the concepts of cloud native architecture and microservice architecture. The most mainstream container runtime Docker recommends that each container only run one main process in its official early documents and tutorials. However, in scientific research and production application systems, the solution of splitting each process into a container is not only labor-intensive and difficult to implement, but also cannot meet the requirements of multi-process priority scheduling, process interdependence, process affinity / anti-affinity scheduling, etc. It is more practical to package multiple processes with coupling, logical dependence and affinity into a container image.

[0004] Although the common methods of managing multiple processes in a single container can solve the problem, they each have their own shortcomings, specifically: 1. Use CMD or ENTRYPOINT in Dockerfile to set the default startup command. However, if the container needs to start multiple processes, process management and restart problems are usually encountered. Each docker image needs to develop and maintain entrypoints, which leads to redundant development work and high operation and maintenance costs. 2. Supervisor process management tools can be used to start and manage multiple processes at the same time and provide a restart mechanism. However, Supervisor increases the complexity of containers and may also introduce additional resource overhead. Deploying Supervisor in Docker images requires the introduction of more dependency packages, which increases the size of container images. Supervisor is repeatedly installed in each image, which consumes more host computing resources and storage resources. Supervisor is a mature process management tool widely used in production environments and has a universal configuration. However, it cannot directly manage processes in containers when deployed on the host. Deploying a supervisor in each container application can achieve self-management of container processes, but there are problems such as complex management and redundant deployment. 3. Use a startup script to execute a script to start multiple processes when starting the container. This method is simple, but lacks advanced management and restart functions. The quality of the script is guaranteed by the developer of each docker image. Summary of the invention

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: In a first aspect, a method for managing the lifecycle of a container system process comprises the following steps: Step 1: Build a container cluster system that covers both stand-alone and cluster modes. In stand-alone mode, remove the centralized control of control plane nodes and directly access working nodes to obtain information. Step 2: Design a configuration management mapping solution from container process management to host process management. Use the Linux kernel's namespace and resource management subsystem to convert the process configuration in the container to the process configuration on the host. The process configuration in the container includes the container name, process name, startup command, execution directory, and other parameters (auto-start, auto-restart, startup wait time, restart times, priority, dependency, log management, etc.). The process configuration on the host includes the host process name, process startup method, and parameters. Step 3: On the host, according to the converted process configuration, start and initialize the corresponding process, set the startup priority (priority) and dependency (depends_on) parameters for processes in different containers and different processes in the same container, solve the coupling management problem between processes, and then allocate resources to the process and initialize the data structure; Step 4: The operating system schedules the process to run on the specified CPU core and memory NUMA node, PCI (network card, GPU, etc.) NUMA node, suspends and resumes the process as needed, and manages priorities and dependencies to optimize resource utilization and performance; Step 5: Continuously monitor the process, define the standard output and error output log paths, and perform log compression and rollback processing to facilitate troubleshooting and performance analysis; Step 6: When a process completes its task or encounters an unrecoverable error, release the resources it occupies and put it into a terminated state to ensure effective use and recovery of system resources.

[0006] Preferably, in step 1, the process of building the container cluster system includes: Install Docker on the target host, build a container cluster system that adapts to both stand-alone and cluster modes, and install Docker Compose; Configure the stand-alone mode, create a new project directory to store the Docker Compose configuration file and other related files, and create a file named docker-compose.yml in the project directory to define the configuration information of the container, including images, port mappings, data volumes, and network modes. In stand-alone mode, directly execute the docker-compose up command in the project directory to start all containers according to the configuration of the docker-compose.yml file, and then directly access the container of the working node to obtain the required information.

[0007] Preferably, in step 2, the process of converting the process configuration in the container to the process configuration on the host includes: Analyze the requirements of processes within the container, including container name, process name, startup command, execution directory, and other parameters (auto-start, auto-restart, startup wait time, restart times, priority, dependencies, and log management, etc.); Parse the process configuration file in the container, extract the required information, and use the mapping rules from container process configuration to host process configuration to convert the process configuration in the container to the process configuration on the host, including converting the container name and process name to the process name on the host, and passing the startup command and execution directory, and then design a configuration management mapping solution from container process management to host process management; Prepare the configuration file on the host. The configuration file on the host contains the converted host process configuration. The configuration file is in INI, YAML, JSON and other formats. Use Linux namespaces (PID, Network, IPC, etc.) to isolate the processes in the container to ensure that they run in an isolated manner on the host. Use cgroups to limit, record, and isolate the physical resources (CPU, memory, I / O, etc.) used by the container process.

[0008] Preferably, the configuration management mapping solution from container process management to host process management specifically includes: Get the container name or ID and the command space where the container is located; Get the process information that needs to be started for each container, including process name, startup command, execution command folder, startup parameters, process management parameters, and log configuration; Based on cluster requirements, globally arrange the coupling relationship between processes, including complex logic such as dependencies and startup priorities; Generate a configuration file that the host can correctly identify.

[0009] Preferably, in step 3, the process of initializing the data structure includes: After parsing the configuration file, create a process dependency graph and set the startup priority (priority) and dependency (depends_on) of the process according to the extracted parameters. The nodes represent the processes and the edges represent the dependencies. The topological sorting algorithm is used to determine the startup order of the processes. Allocate CPU, memory, I / O and other resources to each process based on the priority and dependencies of the process, and use cgroups to limit and record the physical resources used by the process; Create and initialize data structures including memory pools, buffers, and queues for processes, where data structures are used for communication, data storage, and sharing between processes; According to the determined startup order and priority, call the API provided by the system to create a new process instance and start the processes in sequence.

[0010] Preferably, in step 4, the specific process of suspending and resuming the process includes: Identify NUMA (Non-Uniform Memory Access) nodes in the system, including CPU cores, memory, and PCI devices (such as network cards, GPUs), and configure NUMA policies to optimize memory access, reduce latency, and improve performance; Set CPU affinity for the process so that the process runs on the specified CPU core, configure memory affinity so that the process preferentially uses the memory on a specific NUMA node, and allocate the corresponding PCI devices (such as GPU, network card) to the required process; Define different priorities for processes based on business needs and performance goals, clarify dependencies between processes, ensure that dependent processes start before dependent processes, monitor process resource usage and performance indicators in real time, and dynamically adjust process priority and CPU affinity scheduling parameters based on monitoring results; When a process needs to wait for an event (I / O operation completion, resource acquisition, etc.), the process is suspended through an operating system command. After the suspension operation is completed, the process is resumed through an operating system command.

[0011] Preferably, the process of suspending and resuming the process specifically includes: During the execution of a process, if it encounters an event that requires waiting, such as reading disk data (I / O operation), waiting for a resource (such as a lock or semaphore) to be released, or waiting for another process to complete a task, the process triggers the waiting condition, and the operating system converts it from the running state (Running) to the waiting state (Waiting / Blocked) and saves the context of the process (register values, program counter, stack pointer, etc.) so that it can continue from the interruption point when the process resumes execution; The operating system maintains one or more waiting queues to store processes in a waiting state, divides the waiting queues according to the type of events being waited for, and adds the suspended processes to the corresponding waiting queues to wait for the corresponding events to occur; The operating system continuously monitors the events that the processes in the waiting queue depend on. When an event is detected (I / O operation completion, resource availability, etc.), the operating system removes the corresponding process from the waiting queue and changes its state from the waiting state to the ready state (Ready). The operating system maintains one or more ready queues to store processes in the ready state. At this time, the processes in the ready queue are ready to execute, but have not yet been scheduled to run on the CPU. The awakened process is added to the ready queue and waits for the operating system's scheduler to select it for execution; The operating system's scheduler selects a process from the ready queue to execute based on the priority scheduling algorithm. Before the process is scheduled to execute on the CPU, the operating system restores its previously saved context, including register values, program counters, etc. After the context restoration is completed, the process continues to execute from where it last stopped.

[0012] Preferably, in step 5, the process of continuously monitoring the progress includes: Define the log file path for standard output (stdout) and standard error output (stderr) for each process, and set the log format, including timestamp, process ID and log level (such as INFO, ERROR, DEBUG), etc., to facilitate subsequent log analysis; Select and configure a logging system to automatically collect the stdout and stderr output of the process and redirect them to the specified log file. The logging system includes rsyslog, syslog-ng, logrotate, etc. Use the logrotate tool to configure the log rotation strategy to prevent the log file from growing indefinitely. Configure the log rotation tool to compress the log files into .gz files using the gzip compression tool when the log files reach a certain size. Set the log rollback strategy to retain a certain number of historical log files. The old log files are compressed and renamed. Use real-time log monitoring tools to monitor and analyze logs, perform log queries to identify errors, warnings, and other critical events, and analyze logs to identify performance bottlenecks including slow queries and high latency; Configure the alarm system to trigger an alarm when a specific error or abnormal log occurs, and notify the system administrator of the alarm information via email, SMS or instant message.

[0013] Preferably, in step 6, the process of releasing occupied resources includes: The operating system periodically checks the status of each process, or checks the process status when a signal exception event occurs. If a process encounters an unhandled exception, including a division by zero operation or access to an illegal memory address, the operating system captures the corresponding signal and handles it; The process sends a signal to another process through the kill() system call to terminate the target process. The signals include SIGTERM and SIGKILL. SIGTERM is a request to terminate, allowing the process to clean up, and SIGKILL is a forced termination, not allowing the process to clean up. Before a process terminates, it releases the system resources occupied by the process, including memory, file handles, network connections, etc., by calling the release function and resource recovery mechanism provided by the system. After the process terminates, the operating system updates the process status to "Terminated" and removes the entry of the process from the process table, indicating that the process no longer occupies CPU time and no longer participates in process scheduling; The terminated process sends a signal to its parent process to notify the parent process that the process has terminated. The parent process obtains the termination status and resource usage of the child process through the wait() or waitpid() system call to wait for the child process to end and reclaim its resources. The operating system reclaims the CPU time slice, memory used by the process, and I / O resources used by the process, and records the process termination information in the system log, including the termination reason and time. If the process terminates due to an error, detailed error information is recorded.

[0014] In a second aspect, a container system process lifecycle management system is provided, which is used to implement a method for container system process lifecycle management, including a container cluster system, wherein the container cluster system is communicatively connected with a process management scheduling module, a resource allocation and NUMA optimization module, a log management module, a data structure management module, a health monitoring alarm module, and a resource management recovery module, wherein electrical signals are connected between the modules; The process management and scheduling module is used to create, schedule and monitor the execution of processes in the container, ensure that the processes are started and run according to the predetermined priorities and dependencies, and optimize resource utilization and performance; The resource allocation and NUMA optimization module is used to manage and allocate CPU, memory and I / O resources to processes, and perform NUMA optimization to reduce latency, improve performance, and ensure that processes access resources efficiently, especially under the NUMA architecture; The log management module is used to collect, store, compress and analyze the standard output and error output logs of the process, which is convenient for problem troubleshooting, performance monitoring and security auditing, and reduces storage space by log compression; The data structure management module is used to manage the communication mechanism between processes including memory pools, buffers and queues, and initialize data structures to support efficient inter-process communication and data sharing, thereby improving the overall throughput and response speed of the system; The health monitoring and alarm module is used to monitor the health status of the process, including resource usage and performance indicators, and trigger alarms in abnormal situations, timely discover and respond to system problems, reduce system failure time, and improve system reliability and stability; The resource management and recycling module is used to manage the normal termination and abnormal exit of the process, so that the process releases all occupied resources when it ends, avoids resource leakage, ensures that system resources are effectively recycled and reused, and maintains the overall health and performance of the system.

[0015] The present invention provides a method and system for managing the lifecycle of a container system process. It has the following beneficial effects: 1. The method and system for process lifecycle management of the container system improve resource utilization and system performance through fine resource allocation and NUMA optimization. Under the NUMA architecture, by identifying and configuring NUMA nodes, it ensures that processes give priority to using memory on specific NUMA nodes, reduces memory access latency, improves data processing efficiency, and dynamically adjusts CPU and memory resource allocation according to process priorities and dependencies, ensuring that key processes obtain sufficient resources to maintain optimal performance.

[0016] 2. The method and system for process lifecycle management of the container system can enhance the reliability and stability of the system by continuously monitoring and checking the health of the process, timely discovering and responding to process failures or performance bottlenecks. When the process encounters an unrecoverable error, it ensures that the process releases resources and terminates to avoid the risk of resource leakage and system crash. It also tracks the running status of the process through real-time log monitoring tools, quickly locates the source of the problem, and accelerates the fault recovery process.

[0017] 3. The method and system for process lifecycle management of the container system proposes a simple and effective mapping method from the process started in the container to the container process started on the host, which converts the process management problem in the container into a regular management problem of the host. It only needs to deploy the management work once on each host, has strong compatibility with existing tools and consumes less resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A method flow chart of a method for managing the lifecycle of a container system process according to the present invention; Figure 2 It is a flow chart of the present invention; Figure 3 This is a system module diagram of the present invention. DETAILED DESCRIPTION

[0019] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

[0020] The first embodiment, as Figure 1 , Figure 2 As shown, the present invention provides a technical solution: a method for managing the lifecycle of a container system process, comprising the following steps: Step 1: Build a container cluster system covering stand-alone and cluster modes. In stand-alone mode, remove the centralized control of control plane nodes and directly access working nodes to obtain information. Install Docker on the target host, build a container cluster system that adapts to both stand-alone and cluster modes, and install Docker Compose. Docker is the basis for building container clusters. Docker Compose is a container orchestration tool that allows users to define multiple related application containers in a YAML file and start and manage containers through simple commands. Configure the stand-alone mode, create a new project directory, store Docker Compose configuration files and other related files, and create a file named docker-compose.yml in the project directory to define the configuration information of the container, including images, port mappings, data volumes, and network modes. In stand-alone mode, you can directly define the required working nodes (containers) and services in the docker-compose.yml file without defining control plane nodes. In stand-alone mode, directly execute the docker-compose up command in the project directory to start all containers according to the configuration of the docker-compose.yml file, and then directly access the containers of the working nodes to obtain the required information. Step 2: Design a configuration management mapping solution from container process management to host process management. Use the Linux kernel's namespace and resource management subsystem to convert the process configuration in the container into the process configuration on the host. The process configuration in the container includes the container name, process name, startup command, execution directory and other parameters (auto-start, auto-restart, startup waiting time, number of restarts, priority, dependency, log management, etc.). The process configuration on the host includes the host process name, process startup method and parameters. Analyze the needs of the process in the container, including the container name, process name, startup command, execution directory and other parameters (auto-start, auto-restart, startup waiting time, number of restarts, priority, dependency and log management, etc.), parse the process configuration file in the container, extract the required information, and use The mapping rules from container process configuration to host process configuration convert the process configuration in the container into the process configuration on the host, including converting the container name and process name into the process name on the host, and passing the startup command and execution directory. For example, mature process management tools such as supervisor start the process according to the process startup command, and perform subsequent parameter and log management through the process name. The commands used to run the process in the container and the process on the host are different, and the process name and process ID seen from the two perspectives are also different. That is, the host side cannot directly identify the process correctly based on the information seen in the container, and the container cannot sense the process information of other containers, and cannot directly implement multi-process association management between containers. Therefore, a configuration management mapping solution from container process management to host process management is designed; Prepare the configuration file on the host. The configuration file on the host contains the converted host process configuration. The configuration file is in INI, YAML, JSON and other formats. Use Linux namespaces (PID, Network, IPC, etc.) to isolate the processes in the container and ensure that they run in an isolated manner on the host. Use cgroups to limit, record and isolate the physical resources (CPU, memory, I / O, etc.) used by the container process. Furthermore, the configuration management mapping solution from container process management to host process management specifically includes: Get the container name or ID, the command space where the container is located, and the process information that needs to be started for each container, including the process name, startup command, execution command folder, startup parameters, process management parameters, and log configuration. According to cluster requirements, globally arrange the coupling relationship between processes, including complex logic such as dependency and startup priority, and generate a configuration file that the host can correctly identify. Specifically: The format of the host process configuration file in the prior art is as follows. This format file can be used in a container where a process management tool (such as Supervisor) has been deployed: Command: Process startup command C1 (including startup parameters) Folder: Folder Name F1 Process name: The process name in the container N1 Process management users and groups: User U1 Priority: local process priority within the container, P1 Log Management: Log Management Parameters Other parameters (such as restart parameters, monitoring checks, etc.): other parameters; In addition, the configuration management mapping solution from container process management to host process management is used. The format of the container process configuration file is as follows. This format can be used directly on the host, and there is no need to install a process management tool for each container: Start command: container execution command + container namespace + container name + enter folder name F1 + associated symbol + process start command C1 (including start parameters); Example: "docker exec" + "-itn" + "container namespace" + "container name" + "cdF1" + "&&" + "C1"; Folder: Empty Process name: container namespace_container name_process name in container N1 Process management users and groups: User X Priority: Host-wide global process priority Log Management: Log Management Parameters Other parameters (such as restart parameters, monitoring checks, etc.): other parameters; Step 3: On the host, according to the converted process configuration, start and initialize the corresponding process, set the startup priority (priority) and dependency (depends_on) parameters for processes in different containers and different processes in the same container, solve the coupling management problem between processes, and then allocate resources to the process and initialize the data structure. After parsing the configuration file, create a process dependency graph, set the startup priority (priority) and dependency (depends_on) of the process according to the extracted parameters, where the node represents the process and the edge represents the dependency. Use the topological sorting algorithm to determine the startup order of the process. According to the priority and dependency of the process, allocate CPU, memory, I / O and other resources to each process. Use cgroups to limit and record the physical resources used by the process. Create and initialize data structures including memory pools, buffers and queues for the process. The data structure is used for communication, data storage and sharing between processes. According to the determined startup order and priority, call the API provided by the system to create a new process instance and start the process in sequence. Step 4: The operating system schedules the process to run on the specified CPU core and memory NUMA node, PCI (network card, GPU, etc.) NUMA node, suspends and resumes the process as needed, and manages priorities and dependencies to optimize resource utilization and performance, and identify NUMA (Non-Uniform Memory) nodes in the system. NUMA nodes, including CPU cores, memory, and PCI devices (such as network cards and GPUs), and configure NUMA policies to optimize memory access, reduce latency, and improve performance. Set CPU affinity for processes so that they run on specified CPU cores. Configure memory affinity so that processes use memory on specific NUMA nodes first, and allocate corresponding PCI devices (such as GPUs and network cards) to required processes. Define different priorities for processes based on business needs and performance goals, clarify dependencies between processes, ensure that dependent processes start before dependent processes, monitor resource usage and performance indicators of processes in real time, and dynamically adjust the scheduling parameters of priority and CPU affinity in processes based on monitoring results. When a process needs to wait for an event (I / O operation completion, resource acquisition, etc.), suspend the process through operating system commands, and resume the process through operating system commands after the suspend operation is completed. Furthermore, the process of process suspension and resumption specifically includes: During the execution of a process, if it encounters an event that requires waiting, such as reading disk data (I / O operation), waiting for a resource (such as a lock or semaphore) to be released, or waiting for another process to complete a task, the process triggers the waiting condition, and the operating system converts it from the running state (Running) to the waiting state (Waiting / Blocked) and saves the context of the process (register values, program counter, stack pointer, etc.) so that it can continue from the interruption point when the process resumes execution. The operating system maintains one or more waiting queues to store processes in the waiting state, divides the waiting queues according to the type of event being waited for, and adds the suspended process to the corresponding waiting queue, waiting for the corresponding event to occur. The operating system continuously monitors the events that the processes in the waiting queues depend on. , when an event is detected (I / O operation completed, resources available, etc.), the operating system removes the corresponding process from the waiting queue and changes its state from the waiting state to the ready state (Ready). The operating system maintains one or more ready queues to store processes in the ready state. At this time, the processes in the ready queue are ready to execute, but have not yet been scheduled to run on the CPU. The awakened process is added to the ready queue and waits for the operating system's scheduler to select it for execution. The operating system's scheduler selects a process from the ready queue for execution based on the priority scheduling algorithm. Before the process is scheduled to execute on the CPU, the operating system restores its previously saved context, including register values, program counters, etc. After the context restoration is completed, the process continues to execute from where it last stopped; Step 5. Continuously monitor the process, define the standard output and error output log paths, and perform log compression and rollback processing to facilitate troubleshooting and performance analysis. Define the log file paths for standard output (stdout) and standard error output (stderr) for each process, and set the log format, including timestamp, process ID, and log level (such as INFO, ERROR, DEBUG), etc., to facilitate subsequent log analysis. Select and configure the log system to automatically collect the process's stdout and stderr output and redirect it to the specified log file. The log system includes rsyslog, syslog-ng, and logrotate etc. Use the logrotate tool to configure log rotation strategies to avoid unlimited growth of log files. Configure the log rotation tool to compress log files into .gz files when they reach a certain size using the gzip compression tool, and set a log rollback strategy to retain a certain number of historical log files. Old log files are compressed and renamed. Use real-time log monitoring tools to monitor and analyze logs, perform log queries to identify errors, warnings, and other critical events, and analyze logs to identify performance bottlenecks including slow queries and high latency. Configure an alarm system to trigger alarms when specific errors or abnormal logs occur, and notify system administrators of alarm information via email, SMS, or instant messaging. Step 6. When a process completes its task or encounters an unrecoverable error, it releases the resources it occupies and puts it into a terminated state to ensure effective use and recovery of system resources. The operating system periodically checks the status of each process, or checks the process status when a signal exception occurs. When a process encounters an unhandled exception including division by zero or access to an illegal memory address, the operating system captures the corresponding signal and processes it. The process sends a signal to another process through the kill() system call to terminate the target process. The signals include SIGTERM and SIGKILL. SIGTERM is a request to terminate, allowing the process to clean up. SIGKILL is a forced termination, not allowing the process to clean up. Before the process terminates, it is released by calling the release function and resource recovery mechanism provided by the system. The system resources occupied by the process include memory, file handles, network connections, etc. After the process terminates, the operating system updates the process status to "Terminated" and removes the process entry from the process table, indicating that the process no longer occupies CPU time and no longer participates in process scheduling. The terminated process sends a signal to its parent process to notify the parent process that the process has terminated. The parent process obtains the termination status and resource usage of the child process through the wait() or waitpid() system call to wait for the child process to end and reclaim its resources. The operating system reclaims the CPU time slice, the memory used by the process, and the I / O resources used by the process, and records the process termination information in the system log, including the termination reason and time. If the process terminates due to an error, detailed error information is recorded.

[0021] The second embodiment is based on the first embodiment. Figure 3 As shown, the present invention also provides a technical solution: a container system process lifecycle management system, which is used to implement a method for container system process lifecycle management, including a container cluster system, wherein the container cluster system is communicatively connected with a process management scheduling module, a resource allocation and NUMA optimization module, a log management module, a data structure management module, a health monitoring alarm module, and a resource management recovery module, wherein the modules are connected by electrical signals; The process management and scheduling module is used to create, schedule, and monitor the execution of processes in the container, ensuring that processes are started and run according to predetermined priorities and dependencies, and optimizing resource utilization and performance; Resource allocation and NUMA optimization module, which is used to manage and allocate CPU, memory and I / O resources to processes, and perform NUMA optimization to reduce latency, improve performance, and ensure that processes access resources efficiently, especially in the NUMA architecture; The log management module is used to collect, store, compress and analyze the standard output and error output logs of the process, which is convenient for troubleshooting, performance monitoring and security auditing, and reduces storage space by compressing the logs; The data structure management module is used to manage the communication mechanism between processes, including memory pools, buffers, and queues, and initialize data structures to support efficient inter-process communication and data sharing, thereby improving the overall throughput and response speed of the system. Health monitoring and alarm module, which is used to monitor the health status of the process, including resource usage and performance indicators, and trigger alarms in abnormal situations, so as to timely discover and respond to system problems, reduce system failure time, and improve system reliability and stability; The resource management and recycling module is used to manage the normal termination and abnormal exit of the process, so that the process releases all occupied resources at the end, avoids resource leakage, ensures that system resources are effectively recycled and reused, and maintains the overall health and performance of the system.

[0022] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to the conventional means in the field unless otherwise specified and limited.

Claims

1. A method for managing the lifecycle of a container system process, characterized in that: The following steps are involved: Step 1: Build a container cluster system covering stand-alone and cluster modes; Step 2: Design a configuration management mapping solution from container process management to host process management, and use the Linux kernel's namespace and resource management subsystem to convert the process configuration in the container into the process configuration on the host; Step 3: On the host, according to the converted process configuration, start and initialize the corresponding process, set the startup priority and dependency parameters for the processes of different containers and different processes in the same container, and then allocate resources to the process and initialize the data structure; Step 4: The operating system schedules the process to run on the specified CPU core and memory NUMA node, and PCI NUMA node, and suspends and resumes the process as required, as well as manages priority and dependencies; Step 5: Continuously monitor the process, define the standard output and error output log paths, and perform log compression and rollback processing; Step 6: When the process completes its task or encounters an unrecoverable error, release the resources it occupies and put it into the terminated state.

2. A method for managing the lifecycle of a container system process according to claim 1, characterized in that: In step 1, the process of building the container cluster system includes: Install Docker on the target host, build a container cluster system that adapts to both stand-alone and cluster modes, and install Docker Compose; Configure the stand-alone mode, create a new project directory to store the Docker Compose configuration file and other related files, and create a file named docker-compose.yml in the project directory to define the configuration information of the container, including the image, port mapping, data volume, and network mode; In stand-alone mode, directly execute the docker-compose up command in the project directory to start all containers according to the configuration of the docker-compose.yml file, and then directly access the container of the working node to obtain the required information.

3. A method for managing the lifecycle of a container system process according to claim 2, characterized in that: In step 2, the process of converting the process configuration in the container into the process configuration on the host includes: Analyze the requirements of processes within the container, including container name, process name, startup command, execution directory, and other parameters; Parse the process configuration file in the container, extract the required information, and use the mapping rules from container process configuration to host process configuration to convert the process configuration in the container to the process configuration on the host, including converting the container name and process name to the process name on the host, and passing the startup command and execution directory, and then design a configuration management mapping solution from container process management to host process management; Prepare the configuration file on the host. The configuration file on the host contains the converted host process configuration, uses the Linux namespace to isolate the process in the container, and uses cgroups to limit, record, and isolate the physical resources used by the container process.

4. A method for managing the lifecycle of a container system process according to claim 3, characterized in that: The configuration management mapping solution from container process management to host process management specifically includes: Get the container name or ID and the command space where the container is located; Get the process information that needs to be started for each container, including the process name, startup command, execution command folder, startup parameters, process management parameters, and log configuration; Based on cluster requirements, globally orchestrate the coupling relationship between processes, including complex logic of dependencies and startup priorities; Generate a configuration file that the host can correctly identify.

5. A method for managing the lifecycle of a container system process according to claim 4, characterized in that: In step 3, the process of initializing the data structure includes: After parsing the configuration file, a process dependency graph is created. The startup priority and dependency of the process are set according to the extracted parameters. The nodes represent the processes and the edges represent the dependencies. The startup order of the processes is determined using a topological sorting algorithm. Allocate CPU, memory, and I / O resources to each process based on its priority and dependencies, and use cgroups to limit and record the physical resources used by the process. Create and initialize data structures including memory pools, buffers, and queues for processes, where data structures are used for communication, data storage, and sharing between processes; According to the determined startup order and priority, call the API provided by the system to create a new process instance and start the processes in sequence.

6. A method for managing the lifecycle of a container system process according to claim 5, characterized in that: In step 4, the specific process of suspending and resuming the process includes: Identify NUMA nodes in the system, including CPU cores, memory, and PCI devices, and configure NUMA policies to optimize memory access; Set CPU affinity for the process so that the process runs on the specified CPU core, configure memory affinity so that the process preferentially uses the memory on a specific NUMA node, and allocate the corresponding PCI device to the required process; Define different priorities for processes based on business needs and performance goals, clarify dependencies between processes, ensure that dependent processes start before dependent processes, monitor process resource usage and performance indicators in real time, and dynamically adjust process priority and CPU affinity scheduling parameters based on monitoring results; When a process needs to wait for an event, the process is suspended through an operating system command, and after the suspension operation is completed, the process is resumed through an operating system command.

7. A method for managing the lifecycle of a container system process according to claim 6, characterized in that: The process of suspending and resuming the process specifically includes: When a process encounters an event to wait for during execution, the process triggers the waiting condition, and the operating system converts it from the running state to the waiting state and saves the context of the process; The operating system maintains one or more waiting queues to store processes in a waiting state, divides the waiting queues according to the type of events being waited for, and adds the suspended processes to the corresponding waiting queues to wait for the corresponding events to occur; The operating system continuously monitors the events that the processes in the waiting queue depend on. When an event is detected, the operating system removes the corresponding process from the waiting queue and changes its state from the waiting state to the ready state. The operating system maintains one or more ready queues to store processes in the ready state. The awakened process is added to the ready queue and waits for the operating system's scheduler to select it for execution; The operating system's scheduler selects a process from the ready queue to execute based on the priority scheduling algorithm. Before the process is scheduled to execute on the CPU, the operating system restores its previously saved context, including register values ​​and program counter. After the context restoration is completed, the process continues to execute from where it last stopped.

8. A method for managing the lifecycle of a container system process according to claim 7, characterized in that: In step 5, the process of continuously monitoring the progress includes: Define the log file path for standard output and standard error output for each process, and set the log format, including timestamp, process ID and log level; Select and configure a logging system to automatically collect the process's stdout and stderr output and redirect it to a specified log file; Use the logrotate tool to configure the log rotation strategy. Configure the log rotation tool to use the gzip compression tool to compress the log files into .gz files, and set the log rollback strategy to retain historical log files. Use real-time log monitoring tools to monitor and analyze logs, perform log queries to identify errors, warnings, and other critical events, and analyze logs to identify performance bottlenecks including slow queries and high latency; Configure the alarm system to trigger an alarm when a specific error or abnormal log occurs, and notify the system administrator of the alarm information via email, SMS or instant message.

9. A method for managing the lifecycle of a container system process according to claim 8, characterized in that: In step 6, the process of releasing occupied resources includes: The operating system periodically checks the status of each process, or checks the process status when a signal exception event occurs. If a process encounters an unhandled exception, including a division by zero operation or access to an illegal memory address, the operating system captures the corresponding signal and handles it; The process sends a signal to another process through the kill() system call to terminate the target process. The signals include SIGTERM and SIGKILL. SIGTERM is a request to terminate, allowing the process to clean up, and SIGKILL is a forced termination, not allowing the process to clean up. Before a process terminates, it releases the system resources occupied by the process by calling the release function and resource recovery mechanism provided by the system. After the process terminates, the operating system updates the process status to "terminated status" and removes the entry of the process from the process table, indicating that the process no longer occupies CPU time and no longer participates in process scheduling; The terminated process sends a signal to its parent process to notify the parent process that the process has terminated. The parent process obtains the termination status and resource usage of the child process through the wait() or waitpid() system call to wait for the child process to end and reclaim its resources. The operating system reclaims the CPU time slice, memory used by the process, and I / O resources used by the process, and records the process termination information in the system log, including the termination reason and time. If the process terminates due to an error, detailed error information is recorded.

10. A container system process lifecycle management system, used to implement the container system process lifecycle management method according to any one of claims 1 to 9, comprising a container cluster system, characterized in that: The container cluster system is connected in communication with a process management and scheduling module, a resource allocation and NUMA optimization module, a log management module, a data structure management module, a health monitoring and alarm module, and a resource management and recycling module, wherein the modules are connected by electrical signals; The process management and scheduling module is used to create, schedule and monitor the execution of processes in the container; The resource allocation and NUMA optimization module is used to manage and allocate CPU, memory and I / O resources to processes and perform NUMA optimization; The log management module is used to collect, store, compress and analyze the standard output and error output logs of the process; The data structure management module is used to manage the communication mechanism between processes including memory pools, buffers and queues, and initialize data structures; The health monitoring and alarm module is used to monitor the health status of the process, including resource usage and performance indicators, and trigger alarms in abnormal situations; The resource management and recycling module is used to manage the normal termination and abnormal exit of a process, so that the process releases all occupied resources when it ends.

Images