Log storage method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a log storage method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a log file generated during service execution through a service container in a system node; writing the log file into a local directory in a system node mounted by a log directory in the service container through the service container; and reading the log file from a local directory mounted on a source directory in a synchronization container through the synchronization container in the system node, and synchronizing the read log file to a first shared directory in the synchronization container, so that the log file is stored to a storage server mounted on the first shared directory. By adopting the method, the reliability of log storage can be ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and particularly to a log storage method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of containerization technologies, container tools can be used in the cloud platform of a business system to execute corresponding business tasks. Generally, log files are generated during business execution and need to be stored in an external storage device. Generally, the cloud platform can use a storage device that provides Network File System (NFS, a distributed file system protocol) to achieve remote storage of log files in the storage device. However, when the cloud platform transfers data to the storage device, it is greatly affected by the network. Once a network outage or other situations occur, the write operation inside the cloud platform is blocked, and the service cannot continue to be provided, resulting in continuous interruption of log file storage. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a log storage method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the stability of the normal operation of business containers and thus ensure the reliability of log storage.

[0004] In a first aspect, the present application provides a log storage method, which is applied to a system node of a business system and includes:

[0005] Obtain, through a business container in the system node, a log file generated during business execution;

[0006] Write, through the business container, the log file into a local directory in the system node mounted by a log directory in the business container;

[0007] Read, through a synchronization container in the system node, the log file from the local directory mounted by a source directory in the synchronization container, and synchronize the read log file to a first shared directory in the synchronization container, so that the log file is stored in a storage server mounted by the first shared directory.

[0008] In a second aspect, the present application further provides a log storage apparatus, including:

[0009] An obtaining module, configured to obtain, through a business container in the system node, a log file generated during business execution;

[0010] A mounting module, configured to write, through the business container, the log file into a local directory in the system node mounted by a log directory in the business container;

[0011] A storage module, configured to read log files from a local directory mounted by a source directory in a synchronization container through a synchronization container in a system node, and synchronize the read log files to a first shared directory in the synchronization container, so that the log files are stored in a storage server mounted by the first shared directory.

[0012] In a third aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0013] Obtain log files generated during business execution through a business container in a system node;

[0014] Write the log files into a local directory in the system node mounted by a log directory in the business container through the business container;

[0015] Read log files from a local directory mounted by a source directory in a synchronization container through a synchronization container in the system node, and synchronize the read log files to a first shared directory in the synchronization container, so that the log files are stored in a storage server mounted by the first shared directory.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0017] Obtain log files generated during business execution through a business container in a system node;

[0018] Write the log files into a local directory in the system node mounted by a log directory in the business container through the business container;

[0019] Read log files from a local directory mounted by a source directory in a synchronization container through a synchronization container in the system node, and synchronize the read log files to a first shared directory in the synchronization container, so that the log files are stored in a storage server mounted by the first shared directory.

[0020] In a fifth aspect, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0021] Obtain log files generated during business execution through a business container in a system node;

[0022] Write the log files into a local directory in the system node mounted by a log directory in the business container through the business container;

[0023] Read log files from the local directory mounted by the source directory in the synchronization container through the synchronization container in the system node, and synchronize the read log files to the first shared directory in the synchronization container, so that the log files are stored in the storage server mounted by the first shared directory.

[0024] The above log storage method, device, computer device, computer-readable storage medium and computer program product mount the log directory to the local directory in the system node through the business container in the system node, so that during the business execution process of the business container, the generated log files are written into the local directory through the container directory, and the log files can be stored in the local directory through the business container; the synchronization container in the system node includes a source directory and a first shared directory, the source directory is mounted to the local directory in the system node, and the first shared directory is mounted to the storage server, so that the synchronization container reads the log files from the local directory through the source directory, and synchronizes the read log files to the first shared directory for storage in the storage server, and the log files can be stored in the storage server through the synchronization container. Therefore, when the system node stores log files in the storage server, it avoids the situation that when the storage server fails during the process of directly mounting the storage server by the business container to store log files, which causes the business container to be unable to write log files normally and affects the normal business functions of the business container. It realizes the decoupling of the write operations of the business container and the storage server, and can improve the stability of the business container operation; further, it can also avoid data loss caused by log file writing when the storage server fails, and improve the reliability of log storage. Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0026] Figure 1 It is an application environment diagram of the log storage method in an embodiment;

[0027] Figure 2 It is a schematic flowchart of the log storage method in an embodiment;

[0028] Figure 3 It is a schematic flowchart of the log storage steps in an embodiment;

[0029] Figure 4 It is a schematic flowchart of directory management in an embodiment;

[0030] Figure 5 Schematic diagram of container deployment in a system node in an embodiment;

[0031] Figure 6 Structural block diagram of a log storage device in an embodiment;

[0032] Figure 7 Internal structure diagram of a computer device in an embodiment;

[0033] Figure 8 Internal structure diagram of a computer device in another embodiment. Detailed implementation manners

[0034] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] The log storage method provided by the embodiment of the present application can be applied to an application environment as Figure 1 shown. Among them, the business system includes each system node 102, and the system node 102 communicates with the storage server 104 through a network. In an exemplary embodiment, a log file generated during business execution is obtained through a business container in the system node 102; the log file is written into a local directory in the system node mounted by the log directory in the business container through the business container; the log file is read from the local directory mounted by the source directory in the synchronization container through the synchronization container in the system node 102, and the read log file is synchronized to the first shared directory in the synchronization container, so that the log file is stored in the storage server 104 mounted by the first shared directory. Among them, the system node 102 can be a terminal or a server. The data storage system can store the data that the system node 102 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers. The terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0036] In an exemplary embodiment, as Figure 2 shown, a log storage method is provided. Taking the method applied to the Figure 1 system node as an example, the method includes the following steps:

[0037] Step 202, obtain a log file generated during business execution through a business container in the system node.

[0038] Among them, a system node refers to any independent processing unit in a business system, generally a computing device, such as a server, a terminal device, etc. A business system refers to a distributed system composed of various system nodes interconnected through a network to jointly process relevant business requirements. A business container refers to a computing unit in a system node used to execute corresponding business tasks. A log file refers to a record file or a set of files used to record system operation events, which can refer to the record files generated by a business container when executing business tasks.

[0039] Exemplarily, a user terminal can send a business processing request to any system node in the business system. After receiving the business processing request, the system node responds to the business processing request through the business container deployed in the system node, executes the corresponding business task, and outputs the corresponding log file after the business task is completed. The number of business containers deployed in the system node can be one or more.

[0040] In an exemplary embodiment, the number of business containers deployed in the system node can be determined according to the performance metrics of the memory in the system node. Specifically, the number of business containers can be dynamically adjusted according to the remaining capacity of the memory and business requirements. For example, when the remaining capacity of the memory in the system node is greater than a preset threshold and a new business task is received, a new business container is created in the system node according to the container parameters of the existing business containers to execute the new business task; and when the remaining capacity of the memory in the system node is less than the preset threshold and no new business task is received, the business containers in the system node that are not currently executing business tasks are destroyed. Among them, before destroying a business container, the log file generated by the business container needs to be stored.

[0041] Step 204, through the business container, write the log file into the local directory in the system node mounted by the log directory in the business container.

[0042] Among them, the local directory refers to the access directory corresponding to the target storage space in the system node for storing log files. The system node can write to or read log files from the target storage space through the local directory. The log directory refers to a specific directory used by the business container to connect to the target storage space of the system node, which can be understood as the mount point of the target storage space of the system node in the business container. By mounting the target storage space to the log directory of the business container, the content in the target storage space can be accessed by the business container under this directory.

[0043] Exemplarily, a target storage space is pre-allocated in the storage space of the system node to store the log files generated by the service container during service execution, and the target storage space is mounted to the log directory in the service container. Since the local directory in the system node can represent the target storage space, the service container can access the target storage space corresponding to the local directory in the system node through the log directory. Specifically, after the service container completes the service task execution and generates the corresponding log files, the service container writes the log files to the local directory mounted by the log files by accessing the log directory, so that the log files are stored in the target storage space corresponding to the local directory.

[0044] In an exemplary embodiment, the local directory can store the image file and configuration parameters corresponding to the service container. After the service container is destroyed, a service container of the same type as the destroyed service container can be created according to the image file and configuration parameters stored in the local directory to execute the same type of service task as the service task historically executed by the destroyed service container.

[0045] Step 206, read the log files from the local directory mounted by the source directory in the synchronization container through the synchronization container in the system node, and synchronize the read log files to the first shared directory in the synchronization container, so that the log files are stored in the storage server mounted by the first shared directory.

[0046] Among them, the synchronization container refers to an operation unit deployed in the system node for performing the operation of storing the log files in the storage server. The storage server refers to a storage device externally connected to the system node for storing log files. The source directory refers to a specific directory used by the synchronization container to connect to the target storage space of the system node, which can be understood as the mount point of the target storage space of the system node in the synchronization container. By mounting the target storage space to the source directory of the synchronization container, the content in the target storage space can be accessed by the synchronization container under this directory. The first shared directory refers to a specific directory used by the synchronization container to connect to the storage server, which can be understood as the mount point of the storage server in the synchronization container. By mounting the storage server to the first shared directory of the synchronization container, the content in the storage server can be accessed by the synchronization container under this directory.

[0047] Exemplarily, a synchronization container is also deployed in the system node. The synchronization container includes a source directory and a first shared directory. The synchronization container accesses the source directory, determines a log file from the target storage space corresponding to the local directory mounted by the source directory, and then accesses the first shared directory to write the read log file into the first shared directory, so as to store the log file to the storage server mounted by the first shared directory. Among them, the first shared directory in the synchronization container can be one or more. When there are multiple first shared directories, the connected storage servers can be the same or different. For example, multiple first shared directories are respectively the mount points in the synchronization container of different storage spaces in the same storage server. For example, multiple first shared directories are respectively the mount points in the synchronization container of different storage servers.

[0048] In an exemplary embodiment, the local directory of the system node can store the log files that need to be stored and are sent by other system nodes, and store the log files corresponding to other system nodes to the corresponding storage servers through the synchronization container.

[0049] Specifically, the system node can receive a log storage request sent by other system nodes. The log storage request can be a request uniformly sent by other system nodes to each system node of the business system. The log storage request includes the log files that need to be stored and are generated by other system nodes and the other server identifiers of the storage servers corresponding to other system nodes.

[0050] After receiving the log storage request, the system node sends a server query request to the synchronization container according to the other server identifier carried in the log storage request, so that the synchronization container returns the local server identifier corresponding to the mounted storage server to the system node according to the storage server mounted by the first shared directory. The local server identifier can be one or more.

[0051] Then when the system node finds the other server identifier in the local server identifiers, it stores the log files corresponding to other system nodes to the local directory, and determines the mount point of the storage server corresponding to the other server identifier in the synchronization container through the synchronization container, that is, the target shared directory in the first shared directory. Then, the log files corresponding to other system nodes are synchronized to the target shared directory through the synchronization container, so that the log files corresponding to other system nodes are stored to the storage server mounted by the target shared directory, that is, the storage server corresponding to the other server identifier.

[0052] In the above log storage method, in the business container of the system node, the log directory is mounted to the local directory in the system node, so that during the execution of the business by the business container, the generated log files are written into the local directory through the container directory, and the log files can be stored in the local directory through the business container; in the synchronization container of the system node, there are a source directory and a first shared directory. The source directory is mounted to the local directory in the system node, and the first shared directory is mounted to the storage server, so that the synchronization container reads the log files from the local directory through the source directory and synchronizes the read log files to the first shared directory for storage in the storage server, and the log files can be stored in the storage server through the synchronization container. Therefore, when the system node stores log files in the storage server, it avoids the situation that when the business container directly mounts the storage server for log file storage, if the storage server fails, the business container cannot write log files normally, which affects the normal business function of the business container. It realizes the decoupling of the write operations of the business container and the storage server, and can improve the stability of the business container operation; further, it can also avoid data loss caused by log file writing when the storage server fails, and improve the reliability of log storage.

[0053] In an exemplary embodiment, as Figure 3 shown, the local directory includes subdirectories corresponding to at least one business container; the subdirectories include the business logs generated by the corresponding business containers when executing the business; in step 206, after reading the log files from the local directory mounted by the source directory in the synchronization container in the system node and synchronizing the read log files to the first shared directory in the synchronization container, the log storage method further includes:

[0054] Step 302, obtain the current business container list corresponding to the system node, where the current business container list includes the current container information of the business containers currently existing in the system node;

[0055] Step 304, based on the current container information, use the subdirectory corresponding to the destroyed business container in the local directory as the target subdirectory, and add a deletion mark to the target subdirectory;

[0056] Step 306, based on the deletion mark, delete the target subdirectory in the local directory.

[0057] Among them, the current container information refers to the relevant information of the business containers currently existing in the system node, including the container identifier (such as container ip) of the currently existing business containers and the current storage address of the log files generated by the business containers in the system node. The subdirectory refers to the subdirectory in the local directory that stores the log files generated by the business container, that is, the storage address of the log files in the system node.

[0058] Exemplarily, after synchronizing the log files of the service containers to the first shared directory, the system node traverses the storage addresses of the log files generated by the currently existing service containers in the system node in the target storage space to obtain the current container information. The current container information includes the subdirectories corresponding to the currently existing service containers in the local directory (i.e., the storage addresses of the log files in the target storage space), and then sends the current container information to the synchronization container. After receiving the current container information, the synchronization container can traverse each subdirectory in the local directory by accessing the local directory mounted by the source directory, compare each subdirectory in the local directory with each subdirectory in the current container information. When it is detected that a certain subdirectory in the local directory does not exist in the current container information, it means that after a certain service container in the system node is destroyed, the system node traverses the current container information generated by each currently existing service container and will not record the container information of the destroyed service container, while there is still a subdirectory corresponding to the storage address for storing the log files of the destroyed service container in the local directory of the system node. Then, the subdirectory in the local directory that does not exist in the current container information is determined as the target subdirectory corresponding to the destroyed service container, and a deletion mark is added to the target subdirectory in the local directory.

[0059] By accessing the source directory through the synchronization container, the target subdirectory with a deletion mark added in the local directory mounted by the source directory is deleted.

[0060] In this embodiment, by detecting the destroyed service containers according to the current container information and the local directory after synchronizing the log files to the first shared directory, and deleting the target subdirectories corresponding to the destroyed service containers in the local directory, the storage resources of the system node can be saved.

[0061] In an exemplary embodiment, the deletion mark includes a mark duration; step 306, deleting the target subdirectory in the local directory based on the deletion mark includes:

[0062] Traverse the mark duration of the deletion mark corresponding to the target subdirectory in the local directory;

[0063] Determine the target subdirectory with the mark duration reaching the preset duration as the subdirectory to be deleted;

[0064] Delete the subdirectory to be deleted in the local directory.

[0065] Exemplarily, when adding a deletion mark to a target subdirectory, the synchronization container also records the addition time of the deletion mark corresponding to the target subdirectory, and accumulates the mark duration of the deletion mark according to the addition time of the deletion mark. Then, after a preset time period, the synchronization container traverses the local directory mounted by the source directory by accessing the source directory, views the mark durations of the deletion marks corresponding to each target subdirectory in the local directory, and determines the target subdirectories with mark durations reaching the preset duration as the subdirectories to be deleted. Among them, different target subdirectories may correspond to different preset durations, or different preset durations corresponding to different system nodes.

[0066] In an exemplary embodiment, as Figure 4 shown, a process schematic diagram of directory management is provided, which is applied to a synchronization container. Before synchronizing the log file to the first shared directory, the synchronization container calls the API (application interface) of the system node to obtain the container information of the current business container of the system node, such as the container IP list of the business container. The synchronization container determines the business containers that need to perform log file synchronization currently according to the container IP list, reads the log files corresponding to the business containers recorded in the container IP list from the local directory mounted by the source directory by accessing the source directory, and synchronizes the read log files to the first shared directory.

[0067] After the synchronization container synchronizes the read log file to the first shared directory, it performs a storage management operation on the local directory. Specifically, it can detect whether there are destroyed business containers in the system node, traverse each sub-directory of the local directory through the local directory mounted by the source directory, and determine whether each sub-directory is the sub-directory corresponding to the business-related log file. If not, it returns to traverse each sub-directory of the local directory; if so, it obtains the latest current container information and determines whether the sub-directory in the local directory exists in the current container information. If all exist, it returns to traverse each sub-directory of the local directory; if there is a target sub-directory that does not exist, it adds a deletion mark to the target sub-directory in the local directory until all target sub-directories have added deletion marks, and determines that the traversal is completed; then the synchronization container traverses each sub-directory of the local directory again through the local directory mounted by the source directory, checks the addition duration of the deletion mark of the target sub-directory in the local directory, and deletes the sub-directory to be deleted in the target sub-directory whose addition duration reaches the preset duration until all sub-directories to be deleted are deleted, and determines that the traversal is completed; the local directory also includes the sub-directory corresponding to the large file, which represents the non-business-related non-log file generated by the business container. After the synchronization container finishes traversing the sub-directory to be deleted, it traverses each sub-directory of the local directory again through the local directory mounted by the source directory, and deletes the sub-directory corresponding to the large file in the local directory that meets the deletion conditions. The deletion conditions are, for example, that the storage time of the large file exceeds the preset time, or the capacity of the large file exceeds the preset capacity, etc., until all large files that meet the deletion conditions are deleted, and determines that the traversal is completed. The synchronization container exits the storage management operation on the local directory and determines that the storage management operation is completed.

[0068] In this embodiment, after adding a deletion mark to the target sub-directory, traversing the addition duration of the deletion mark of the target sub-directory again and deleting the target sub-directory whose addition duration reaches the preset duration, that is, deleting after a period of time, can avoid accidental deletion of the target sub-directory and ensure the data security of the local directory.

[0069] In an exemplary embodiment, the log storage method further includes:

[0070] When the storage server fails, continue to execute the step of writing the log file into the local directory of the system node mounted by the log directory in the business container through the business container;

[0071] Stop the step of the synchronization container in the system node reading the log file from the local directory mounted by the source directory in the synchronization container and synchronizing the read log file to the first shared directory in the synchronization container until the storage server resumes normal.

[0072] Exemplarily, a monitoring port for the storage server is also provided in the synchronization container, which is used to receive the operating status of the storage server. When a failure of the storage server is detected through the detection port, the step of continuing to write the log file into the local directory of the system node mounted by the log directory in the service container is executed through the service container. Then, a pause signal is sent to the synchronization container to stop the step of reading the log file from the local directory mounted by the source directory in the synchronization container in the system node and synchronizing the read log file to the first shared directory in the synchronization container until the storage server resumes normal operation.

[0073] In this embodiment, when the storage server fails, by interrupting the synchronization container to intercept the impact of the storage server failure, the service container can continue to execute the operation of writing to the local directory, ensuring the stable operation of the service container.

[0074] In an exemplary embodiment, the system node further includes a management container; the log storage method further includes:

[0075] When the current time reaches the target time, the expired log files in the storage server mounted by the second shared directory in the management container are deleted through the management container.

[0076] The management container refers to a processing unit additionally deployed in the system node for managing the log files in the storage server.

[0077] Exemplarily, a management container is also deployed in the system node. The management container includes a second shared directory, which is a specific directory for the management container to connect to the storage server and can be understood as the mount point of the storage server in the management container. By mounting the storage server to the second shared directory of the management container, the content in the storage server can be accessed by the management container under this directory. When the current time of the system node reaches the target time, the storage server mounted by the second shared directory is accessed through the management container, the file generation date of each log file stored in the storage server is traversed, and the log file whose file generation date exceeds the preset date is determined as the expired log file, and the expired log files in the storage server are deleted.

[0078] In an exemplary embodiment, the management container can be deployed in any randomly selected system node among the various system nodes of the service system, or the system node with the highest privilege level can be selected according to the privilege levels of the various system nodes in the service system to deploy the management container. The management container can be dynamically deployed according to the change of the privilege level of the system node. For example, when the system node where the management container is located detects that the privilege level of the target system node in other system nodes exceeds its own privilege level, the container installation information is generated according to the mirror file and configuration information corresponding to the management container, and the container installation information is sent to the target system node and the local management container is destroyed, so that the target system node creates a management container according to the container installation information and mounts the corresponding storage server.

[0079] In this embodiment, by regularly deleting the expired log files in the storage server through the management container, the storage resources of the storage server can be saved.

[0080] In an exemplary embodiment, the log storage method further includes:

[0081] Obtain a log query request; the log query request carries a target system node identifier corresponding to the target system node;

[0082] Based on the log query request, find the node directory corresponding to the target system node identifier from the storage server mounted on the third shared directory in the system node;

[0083] Obtain the log query result corresponding to the log query request based on the node directory.

[0084] Exemplarily, a third shared directory is further set in the system node. The third shared directory is a specific directory for the system node to connect to the storage server, which can be understood as the mount point of the storage server in the system node. By mounting the storage server to the second shared directory of the system node, the content in the storage server can be accessed by the system node under this directory. The system node can access the storage server through the second shared directory and provide a query service to the user side.

[0085] The system node receives a log query request sent by the user terminal, and the log query request carries the target system node identifier corresponding to the target system node. The system node compares the target system node identifier with its own system node identifier. When the target system node identifier is consistent with the system node's own system node identifier, the system node searches in the local directory according to the log query request and returns the log query result to the user terminal; when the target system node identifier is inconsistent with the system node's own system node identifier, the system node, according to the log query request, accesses the third shared directory and searches for the node directory corresponding to the target system node identifier in the storage server mounted by the third shared directory, obtains the log query result corresponding to the log query request according to the node directory, and then returns the log query result to the user terminal.

[0086] In this embodiment, the third shared directory can be used to query the node directory of other system nodes, improving the data query efficiency.

[0087] In an exemplary embodiment, after step 204, by using the service container to write the log file into the local directory of the system node mounted by the log directory in the service container, the method further includes:

[0088] The system node reads the log file from the local directory and synchronizes the read log file to the third shared directory so that the log file is stored in the storage server mounted by the third shared directory.

[0089] Exemplarily, when the synchronization container is not deployed in the system node, the system node can write the log file to the third shared directory, that is, after the system node reads the log file from the local directory, it synchronizes the read log file to the third shared directory so that the read log file is stored in the storage server mounted by the third shared directory. Among them, the system node can perform the read operation of reading the log file from the local directory and the write operation of writing the log file to the third shared directory by using the binary startup parameter, without the need to execute through the synchronization container, which can achieve the flexibility of log file processing.

[0090] In an exemplary embodiment, as Figure 5As shown in the figure, a schematic diagram of container deployment in a system node is provided. Among them, the business container can be a Java container written in a Java program, which is used to execute corresponding business tasks. The business system can refer to a distributed system involving cloud computing. The system node can be a K8s node running a Java cluster, and the containers deployed on it can be created through K8s (Kubernetes, an open-source container orchestration platform). The log files generated by the Java container can include GC logs (detailed records of the garbage collection process output by the virtual machine, including the current log and rotated logs, named with the characteristic of gc_*.log.*), business logs (including the current log and rotated logs, named with the characteristic of *.log, *.log.gz), OOM dump files (files containing data such as module information, thread information, stack call information, and exception information during program operation, named with the characteristic of *.hprof), etc. The storage server can refer to an NFS (a network shared storage protocol) server, which can provide storage services for the NFS protocol.

[0091] A log management application is pre-deployed in the system node to control the log synchronization of the synchronized containers. The container kd-log-helper is the synchronized container, and the container kd-log-cleaner is the management container. The container kd-log-helper is deployed in the form of a daemonset (a controller in Kubernetes used to ensure that one Pod (container) replica runs on each (or specific) node) on each system node running the java cluster, and can provide the following functions: mount the NFS log disk of the NFS server, that is, the first shared directory (such as Figure 5 the target directory (NFS) in); synchronize the log files (including log files with the naming characteristics of *.log, *.log.gz, gc.*.log, *.hprof, etc.) on the system node to the synchronization process of the NFS log disk using rsync (execute the command); provide a monitoring collection point port (9144) for the NFS server, that is, the monitoring port. The container kd-log-cleaner is deployed in the form of a cronjob (a tool used to execute tasks at specific time intervals), and its main function is to periodically clear expired log files (for example: execute at 2 am every day, and the retention period is 7 days).

[0092] K8s node0 (node 0), K8s node1 (node 1), K8s nodeN (node N) represent system nodes, and each system node deploys its own business containers (such as Figure 5 the Java container instances in: pod-0, pod-1, pod-N), and local directories (such as Figure 5 the host directory in). Figure 5The arrow in it indicates the mounting direction. For example, the log directory of the business container pod-0 is mounted on the host directory. The business container pod-0 can access the host directory by accessing the log directory. Specifically, it can be mounted through the hostpath host disk mapping or through the mount command. The NFS directory represents the third shared directory. The NFS directory in the container cronjob: kd-log-cleaner represents the second shared directory. The NFS / NAS server represents the storage server.

[0093] Specifically, the Java container mounts the log directory where the log files are written to the mount point A (host directory) on the host (system node). When the Java container is running, it writes logs or dump files to the log directory. The name or path of the written file contains the concatenated part of the nodeIP (node ip) and podIP (container ip) to avoid conflicts with files generated by other Java containers. The source directory A in the container kd-log-helper is mounted to the mount point A (host directory) of the Java container. The target directory B in the container kd-log-helper is mounted to the mount point B (storage area in the storage server) of the NFS server, and the content of the directory "A / java container path" is synchronously copied to the directory "B / java container path" in real time. And when the Java container is destroyed by the k8s cluster, the content on the host mount point "A / java container path" is automatically deleted, and the synchronization to B is stopped. The container kd-log-helper also provides a collection point port that meets the prometheus standard for monitoring the NFS server. The container kd-log-cleaner container mounts the NFS mount point B and implements the cleaning of log files that exceed the retention time limit according to a certain time retention policy.

[0094] When there is an NFS network failure or an NFS server failure, the synchronization process of the container kd-log-helper will be blocked and suspended. The abnormal state of the NFS server can be read through prometheus monitoring, prompting the operation and maintenance personnel to intervene. At the same time, since the business process of the Java container does not directly mount the NFS mount point, it will not be affected. After the failure is eliminated, kd-log-helper resumes synchronization.

[0095] In this embodiment, by using a synchronization container to mount the NFS server, the decoupling of network storage and the disk on which the business container depends during runtime is achieved. It realizes that when the network storage service (NFS) is unavailable, it does not affect the stability of the business container during runtime. At the same time, through the management container with management functions, the cleaning of overdue and oversized log files and other tasks are realized.

[0096] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0097] Based on the same inventive concept, an embodiment of the present application also provides a log storage device for implementing the above-mentioned log storage method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the log storage device provided below can refer to the limitations on the log storage method in the foregoing, and will not be repeated here.

[0098] In an exemplary embodiment, as Figure 6 shown, a log storage device 600 is provided, including: an acquisition module 602, a mounting module 604, and a storage module 606, where:

[0099] The acquisition module 602 is configured to acquire a log file generated during business execution through a business container in a system node;

[0100] The mounting module 604 is configured to write the log file into a local directory in the system node mounted by the log directory in the business container through the business container;

[0101] The storage module 606 is configured to read the log file from the local directory mounted by the source directory in the synchronization container through the synchronization container in the system node, and synchronize the read log file to the first shared directory in the synchronization container, so that the log file is stored in the storage server mounted by the first shared directory.

[0102] In an exemplary embodiment, the local directory includes subdirectories corresponding to at least one business container; the subdirectories include business logs generated by the corresponding business containers when executing business; the log storage device 600 is further configured to:

[0103] Obtain the current list of business containers corresponding to the system nodes. The current list of business containers includes the current container information of the business containers currently existing in the system nodes. Based on the current container information, in the local directory, use the subdirectory corresponding to the destroyed business container as the target subdirectory, and add a deletion mark to the target subdirectory. Based on the deletion mark, delete the target subdirectory in the local directory.

[0104] In an exemplary embodiment, the deletion mark includes a marking duration. The log storage device 600 is further configured to:

[0105] Traverse the marking durations of the deletion marks corresponding to the target subdirectories in the local directory. Determine the target subdirectories whose marking durations reach the preset duration as the subdirectories to be deleted. Delete the subdirectories to be deleted in the local directory.

[0106] In an exemplary embodiment, the log storage device 600 is further configured to:

[0107] When a failure occurs in the storage server, continue to execute the step of writing the log file into the local directory in the system node mounted by the log directory in the business container through the business container. Stop the step of reading the log file from the local directory mounted by the source directory in the synchronization container in the system node and synchronizing the read log file to the first shared directory in the synchronization container until the storage server returns to normal.

[0108] In an exemplary embodiment, the system node further includes a management container. The log storage device 600 is further configured to:

[0109] When the current moment reaches the target moment, delete the expired log files in the storage server mounted by the second shared directory in the management container through the management container.

[0110] In an exemplary embodiment, the log storage device 600 is further configured to:

[0111] Obtain a log query request. The log query request carries the target system node identifier corresponding to the target system node. Based on the log query request, in the storage server mounted by the third shared directory in the system node, search for the node directory corresponding to the target system node identifier. Obtain the log query result corresponding to the log query request based on the node directory.

[0112] In an exemplary embodiment, the log storage device 600 is further configured to:

[0113] Read the log file from the local directory through the system node, and synchronize the read log file to the third shared directory so that the log file is stored in the storage server mounted by the third shared directory.

[0114] Each module in the above-mentioned log storage device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0115] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as log files. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a log storage method.

[0116] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 8As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. The computer program, when executed by the processor, implements a log storage method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0117] Those skilled in the art can understand that Figures 7 - 8 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0118] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0119] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0120] In an embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0122] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited thereto.

[0123] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0124] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several variations and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A log storage method, characterized in that, The method is applied to a system node of a business system; the method includes: Obtain a log file generated during business execution through a business container in the system node; Write the log file into a local directory in the system node mounted by a log directory in the business container through the business container; Read the log file from the local directory mounted by a source directory in the synchronization container through the synchronization container in the system node, and synchronize the read log file to a first shared directory in the synchronization container, so that the log file is stored in a storage server mounted by the first shared directory.

2. The method according to claim 1, wherein The local directory includes subdirectories corresponding to at least one business container; the subdirectories include business logs generated by the corresponding business containers when executing business; after reading the log file from the local directory mounted by the source directory in the synchronization container in the system node and synchronizing the read log file to the first shared directory in the synchronization container, the method further includes: Obtain a current business container list corresponding to the system node, where the current business container list includes current container information of business containers currently existing in the system node; Based on the current container information, use a subdirectory corresponding to a destroyed business container in the local directory as a target subdirectory, and add a deletion mark to the target subdirectory; Delete the target subdirectory in the local directory based on the deletion mark.

3. The method according to claim 2, wherein The deletion mark includes a mark duration; deleting the target subdirectory in the local directory based on the deletion mark includes: Traverse the mark durations of the deletion marks corresponding to the target subdirectories in the local directory; Determine a target subdirectory with a mark duration reaching a preset duration as a subdirectory to be deleted; Delete the subdirectory to be deleted in the local directory.

4. The method according to claim 1, wherein The method further includes: When a failure occurs in the storage server, continue to execute the step of writing the log file into the local directory in the system node mounted by the log directory in the business container through the business container; Stop the step of reading the log file from the local directory mounted by the source directory in the synchronization container in the system node and synchronizing the read log file to the first shared directory in the synchronization container until the storage server resumes normal operation.

5. The method according to claim 1, wherein The system node further includes a management container; the method further includes: When the current moment reaches a target moment, delete expired log files in a storage server mounted by a second shared directory in the management container through the management container.

6. The method according to claim 1, wherein The method further includes: Obtain a log query request; the log query request carries a target system node identifier corresponding to a target system node; Based on the log query request, search for a node directory corresponding to the target system node identifier in a storage server mounted by a third shared directory in the system node; Obtain a log query result corresponding to the log query request based on the node directory.

7. The method according to claim 6, wherein After writing the log file to the local directory in the system node mounted by the log directory in the service container through the service container, the method further includes: Reading the log file from the local directory through the system node, and synchronizing the read log file to the third shared directory, so that the log file is stored in the storage server mounted by the third shared directory.

8. A log storage device, characterized in that, The device includes: An acquisition module, configured to acquire a log file generated during service execution through a service container in a system node; A mounting module, configured to write the log file to a local directory in the system node mounted by the log directory in the service container through the service container; A storage module, configured to read the log file from the local directory mounted by the source directory in the synchronization container through a synchronization container in the system node, and synchronize the read log file to a first shared directory in the synchronization container, so that the log file is stored in the storage server mounted by the first shared directory.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images