A method and apparatus for partitioning logs of a system

Abstract

The application provides a method and device for splitting a system log, and establishes a hard link between a current source log file and a current backup log file. The hard link allows a file to have multiple valid path names, and the file with the established hard link can realize automatic backup. The function and characteristic of the real-time synchronous backup file of the hard link are used to realize real-time synchronization of the log content written to the current source log file to the current backup log file. When splitting the log, the step of copying the current source log file is not needed, and loss of the log data being written during the copying process is avoided. Furthermore, since the current source log file and the current backup file point to the same index node, the log content is continuously written to the current backup log file during the deletion process of the current source log file and the creation process of the next source log file, loss of the log is avoided, the data integrity is ensured, and the reliability and stability of the system are ensured.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and more specifically, to a method and apparatus for segmenting system logs. Background Technology

[0002] With the rapid development of software technology and the internet industry, system logs are crucial for system monitoring and problem localization in our environment. Therefore, a large volume of logs is inevitable in production environments, and the files storing these logs will gradually become very large, putting significant pressure on system operation and potentially creating performance bottlenecks. Log splitting can effectively control and prevent the generation of excessively large log files. One log splitting strategy involves copying the source log file and then clearing it after copying. This operation is time-consuming and may be interrupted by system failures during the copying process. Furthermore, log data being written may be lost during the copying process, compromising system reliability and stability. Summary of the Invention

[0003] The purpose of this application is to provide a system log segmentation method and apparatus to solve the problem that existing log segmentation requires copying the source log file and then clearing the source log file after copying. This operation is time-consuming and may be interrupted by system failure during the copying of the log file. Furthermore, log data being written may be lost during the copying process, which cannot guarantee the reliability and stability of the system.

[0004] This application provides a method for segmenting system logs, applied to a processor, comprising:

[0005] Create the current backup log file and establish a hard link between the current backup log file and the current source log file;

[0006] Log content is continuously written to the current source log file, and the written log content is synchronously sent to the current backup log file through the current hard link;

[0007] Until the current source log file is determined to have contained enough log content to meet a preset condition, the following steps are executed:

[0008] Create the next backup log file;

[0009] Delete the current source log file to remove the current hard link;

[0010] Establish a hard link between the next source log file and the next backup log file;

[0011] Subsequent log content will be continuously written to the next source log file, and the written log content will be synchronized to the next backup log file through the next hard link.

[0012] The above technical solution is a log segmentation method that utilizes hard links to create files. A hard link is established between the current source log file and the current backup log file. The purpose of a hard link is to allow a file to have multiple valid pathnames, enabling automatic file backup. Leveraging the real-time synchronization feature of hard links, log content written to the current source log file is synchronized to the current backup log file in real time. During log segmentation, there is no need to copy the current source log file, avoiding the loss of log data being written during the copying process. Furthermore, since the current source log file and the current backup file point to the same inode, log content continues to be written to the current backup log file during the deletion of the current source log file and the creation of the next source log file, preventing log loss, ensuring data integrity, and guaranteeing system reliability and stability.

[0013] In some alternative implementations, the current hard link includes a connection where the current backup log file and the current source log file are linked by pointing to the same inode; the next hard link includes a connection where the next backup log file and the next source log file are linked by pointing to the same inode.

[0014] In the above technical solution, a hard link refers to a connection established through inodes. In the Linux file system, files stored in disk partitions are assigned inode numbers. A hard link is a connection where multiple filenames point to the same inode. The purpose of hard links is typically to allow a file to have multiple valid pathnames, enabling users to create hard links to important files to prevent accidental deletion. Deleting one file within a hard link does not affect the connections to the others. Therefore, during the deletion of the current source log file and the creation of the next source log file, log content continues to be written to the current backup log file, preventing log loss.

[0015] In some alternative implementations, after deleting the current source log file, the following steps are also included:

[0016] The system will automatically generate the next source log file.

[0017] In the above technical solution, after the current source log file of the system is deleted, the next source log file will be automatically generated. During the deletion process of the current source log file and the creation process of the next source log file, the log content will continue to be written to the current backup log file, thus avoiding log loss.

[0018] In some optional implementations, the system automatically generates a first source log file and a second source log file;

[0019] When the first source log file is used as the current source log file, delete the current source log file, including:

[0020] Delete the first source log file and use the second source log file as the next source log file.

[0021] In the above technical solution, the system has at least a first source log file and a second source log file. When the first source log file is deleted as the current source log file, the system will switch to writing the log content to the second source log file, thus avoiding the problem that the log content cannot be written to the source log file during the deletion process of the current source log file and the creation process of the next source log file.

[0022] In some optional implementations, the preset conditions include a preset time period and / or a preset file size;

[0023] Determine if the log content already written to the current source log file meets preset conditions, including:

[0024] Determine that the time span of the log content already written to the current source log file has reached a preset time period;

[0025] And / or, determine if the current source log file size has reached the preset file size.

[0026] This application provides a system log segmentation device, comprising:

[0027] The creation module is used to create the current backup log file and establish a hard link between the current backup log file and the current source log file;

[0028] The synchronous write module is used to continuously write log content to the current source log file and synchronously write the log content to the current backup log file through the current hard link;

[0029] The judgment module is used to determine whether the log content already written in the current source log file has met the preset conditions; if so, the creation module and the synchronous writing module are used to write the subsequent log content to the next source log file.

[0030] The creation module is also used for: creating the next backup log file; deleting the current source log file to remove the current hard link; and creating a hard link between the next source log file and the next backup log file.

[0031] The synchronous write module is also used to: continuously write subsequent log content to the next source log file, and synchronize the written log content to the next backup log file through the next hard link.

[0032] The log segmentation device in the above technical solution includes a creation module, a synchronous writing module, and a judgment module. The creation module establishes a hard link between the current source log file and the current backup log file. Hard links allow a file to have multiple valid pathnames, enabling automatic file backup. Utilizing the real-time synchronization feature of hard links, the synchronous writing module ensures that log content written to the current source log file is synchronized to the current backup log file in real time. During log segmentation, there is no need to copy the current source log file, avoiding data loss during the copying process. The judgment module determines whether the preset conditions for log segmentation are met. After segmentation, subsequent log content is written to the next source log file. Furthermore, since the current source log file and the current backup file point to the same inode, log content continues to be written to the current backup log file during the deletion of the current source log file and the creation of the next source log file, preventing log loss, ensuring data integrity, and guaranteeing system reliability and stability.

[0033] In some alternative implementations, the system is used to automatically generate the next source log file when the current source log file is deleted.

[0034] In the above technical solution, after the current source log file of the system is deleted, the next source log file will be automatically generated. During the deletion process of the current source log file and the creation process of the next source log file, the log content will continue to be written to the current backup log file, thus avoiding log loss.

[0035] In some alternative implementations, the system is used to generate a first source log file and a second source log file;

[0036] When the first source log file is deleted as the current source log file, the second source log file will be used as the next source log file.

[0037] In the above technical solution, the system has at least a first source log file and a second source log file. When the first source log file is deleted as the current source log file, the system will switch to writing the log content to the second source log file, thus avoiding the problem that the log content cannot be written to the source log file during the deletion process of the current source log file and the creation process of the next source log file.

[0038] An electronic device provided in this application includes a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, perform any of the methods described above.

[0039] This application provides a computer-readable storage medium storing a computer program, which is executed by a processor as described above. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating the steps of a system log segmentation method provided in this application embodiment;

[0042] Figure 2 A functional block diagram of a log segmentation device for a system provided in this application embodiment;

[0043] Figure 3 This application illustrates one possible structure of an electronic device provided in an embodiment of the present application.

[0044] Icons: 1-Creation module, 2-Synchronous write module, 3-Judgment module, 41-Processor, 42-Memory, 43-Communication interface, 44-Communication bus. Detailed Implementation

[0045] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0046] Please refer to Figure 1 , Figure 1 A flowchart illustrating the steps of a system log segmentation method provided in this application embodiment, applied to a processor, includes:

[0047] Step 100: Create the current backup log file and establish a hard link between the current backup log file and the current source log file;

[0048] Step 200: Continuously write the log content to the current source log file, and synchronize the written log content to the current backup log file through the current hard link;

[0049] Step 300: Determine whether the log content already written to the current source log file meets the preset conditions. If so, proceed to step 400.

[0050] Step 400: Create the next backup log file; delete the current source log file to remove the current hard link; create the next hard link between the next source log file and the next backup log file;

[0051] Step 500: Continue writing subsequent log content to the next source log file, and synchronize the written log content to the next backup log file through the next hard link.

[0052] The above technical solution is a log segmentation method that utilizes hard links to create files. A hard link is established between the current source log file and the current backup log file. The purpose of a hard link is to allow a file to have multiple valid pathnames, enabling automatic file backup. Leveraging the real-time synchronization feature of hard links, log content written to the current source log file is synchronized to the current backup log file in real time. During log segmentation, there is no need to copy the current source log file, avoiding the loss of log data being written during the copying process. Furthermore, since the current source log file and the current backup file point to the same inode, log content continues to be written to the current backup log file during the deletion of the current source log file and the creation of the next source log file, preventing log loss, ensuring data integrity, and guaranteeing system reliability and stability.

[0053] In some alternative implementations, the current hard link includes a connection where the current backup log file and the current source log file are linked by pointing to the same inode; the next hard link includes a connection where the next backup log file and the next source log file are linked by pointing to the same inode.

[0054] In this embodiment, a hard link refers to a connection established through inodes. In the Linux file system, files stored in disk partitions are assigned inode numbers. A hard link is a connection where multiple filenames point to the same inode. The purpose of hard links is typically to allow a file to have multiple valid pathnames, enabling users to create hard links to important files and prevent accidental deletion. Deleting one file within a hard link does not affect the connections to the others. Therefore, during the deletion of the current source log file and the creation of the next source log file, log content continues to be written to the current backup log file, preventing log loss.

[0055] In some alternative implementations, after deleting the current source log file in step 400, the system may automatically generate the next source log file.

[0056] In this embodiment of the application, after the current source log file of the system is deleted, the next source log file will be automatically generated. During the deletion process of the current source log file and the creation process of the next source log file, the log content will continue to be written to the current backup log file to avoid log loss.

[0057] In some optional implementations, the system automatically generates a first source log file and a second source log file; when the first source log file is used as the current source log file, step 400 deletes the current source log file, including: deleting the first source log file and using the second source log file as the next source log file.

[0058] In this embodiment of the application, the system has at least a first source log file and a second source log file. When the first source log file is deleted as the current source log file, the system will switch to writing the log content to the second source log file, thus avoiding the problem that the log content cannot be written to the source log file during the deletion process of the current source log file and the creation process of the next source log file.

[0059] In some optional implementations, the preset conditions include a preset time period and / or a preset file size;

[0060] Accordingly, determining that the log content already written to the current source log file meets the preset conditions includes: determining that the time span of the log content already written to the current source log file reaches the preset time period; and / or determining that the file size of the current source log file reaches the preset file size.

[0061] Please refer to Figure 2 , Figure 2 This application provides a functional block diagram of a system log segmentation device, which includes a creation module 1, a synchronous writing module 2, and a judgment module 3.

[0062] The system comprises three modules: Module 1 (Creation) creates the current backup log file and establishes a hard link between it and the current source log file. Module 2 (Synchronous Writing) continuously writes log content to the current source log file and synchronizes the written log content to the current backup log file via the hard link. Module 3 (Judgment) determines whether the log content already written to the current source log file meets preset conditions; if so, it uses Module 1 and Module 2 to write subsequent log content to the next source log file. Module 1 also creates the next backup log file, deletes the current source log file to remove the hard link, and establishes a next hard link between the next source log file and the next backup log file. Module 2 continuously writes subsequent log content to the next source log file and synchronizes the written log content to the next backup log file via the next hard link.

[0063] In this embodiment, the log segmentation device includes a creation module 1, a synchronous writing module 2, and a judgment module 3. The creation module 1 establishes a hard link between the current source log file and the current backup log file. A hard link allows a file to have multiple valid pathnames, enabling automatic file backup. Utilizing the real-time synchronization feature of hard links, the synchronous writing module 2 synchronizes log content written to the current source log file to the current backup log file in real time. During log segmentation, there is no need to copy the current source log file, avoiding data loss during the copying process. The judgment module 3 determines whether the preset conditions for log segmentation are met. After segmentation, subsequent log content is written to the next source log file. Furthermore, since the current source log file and the current backup file point to the same inode, log content continues to be written to the current backup log file during the deletion of the current source log file and the creation of the next source log file, preventing log loss, ensuring data integrity, and guaranteeing system reliability and stability.

[0064] In some optional implementations, the system is used to automatically generate a next source log file when the current source log file is deleted. In this embodiment, after the current source log file is deleted, the system automatically generates a next source log file, and during the deletion process of the current source log file and the creation process of the next source log file, the log content is still continuously written to the current backup log file, thus avoiding log loss.

[0065] In some optional implementations, the system generates a first source log file and a second source log file; when the first source log file is deleted as the current source log file, the second source log file is used as the next source log file. In this embodiment, the system has at least a first source log file and a second source log file. When the first source log file is deleted as the current source log file, the system switches to writing the log content to the second source log file, avoiding the problem that the log content cannot be written to the source log file during the deletion process of the current source log file and the creation process of the next source log file.

[0066] Figure 3 This illustration shows a possible structure of an electronic device provided in an embodiment of this application. (Refer to...) Figure 3 The electronic device includes a processor 41, a memory 42, and a communication interface 43, which are interconnected and communicate with each other via a communication bus 44 and / or other forms of connection mechanism (not shown).

[0067] The memory 42 includes one or more (only one is shown in the figure), which may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 41 and other possible components may access the memory 42 to read and / or write data therein.

[0068] Processor 41 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 41 can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Network Processor (NP), or other conventional processors; it can also be a special-purpose processor, including a Neural-network Processing Unit (NPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Furthermore, when there are multiple processors 41, some can be general-purpose processors and others can be special-purpose processors.

[0069] Communication interface 43 includes one or more (only one is shown in the figure) that can be used to communicate directly or indirectly with other devices to exchange data. Communication interface 43 may include interfaces for wired and / or wireless communication.

[0070] One or more computer program instructions may be stored in memory 42, and processor 41 may read and run these computer program instructions to implement the methods provided in the embodiments of this application.

[0071] Understandable. Figure 3 The structure shown is for illustrative purposes only; the electronic device may also include structures that are more complex than those shown. Figure 3 The more or fewer components shown, or having the same Figure 3 The different structures shown. Figure 3 The components shown can be implemented using hardware, software, or a combination thereof. Electronic devices may be physical devices, such as PCs, laptops, tablets, mobile phones, servers, embedded devices, etc., or they may be virtual devices, such as virtual machines, virtualized containers, etc. Furthermore, electronic devices are not limited to a single device; they can also be a combination of multiple devices or a cluster of a large number of devices.

[0072] This application also provides a computer-readable storage medium storing computer program instructions. These computer program instructions are read and executed by a computer's processor to perform the method provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 3 Memory 42 in electronic devices.

[0073] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0074] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0075] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0076] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0077] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for segmenting system logs, characterized in that, Applied to processors, including: Create the current backup log file and establish a hard link between the current backup log file and the current source log file; Log content is continuously written to the current source log file, and the log content written is synchronously written to the current backup log file through the current hard link; Until the log content already written to the current source log file is determined to meet the preset conditions, the following steps are executed: Create the next backup log file; Delete the current source log file to remove the current hard link; Establish a hard link between the next source log file and the next backup log file; Subsequent log content will be continuously written to the next source log file, and the written log content will be synchronized to the next backup log file through the next hard link.

2. The method as described in claim 1, characterized in that, in, The current hard link includes the connection method in which the current backup log file and the current source log file are connected by pointing to the same inode; the next hard link includes the connection method in which the next backup log file and the next source log file are connected by pointing to the same inode.

3. The method as described in claim 1, characterized in that, After deleting the current source log file, the process also includes: The system automatically generates the next source log file.

4. The method as described in claim 1, characterized in that, The system automatically generates a first source log file and a second source log file; When the first source log file is used as the current source log file, deleting the current source log file includes: Delete the first source log file and use the second source log file as the next source log file.

5. The method as described in claim 1, characterized in that, in, The preset conditions include a preset time period and / or a preset file size; The determination that the log content already written to the current source log file meets the preset conditions includes: Determine that the time span of the log content already written in the current source log file has reached a preset time period; And / or, determine that the current source log file size has reached a preset file size.

6. A system log segmentation device, characterized in that, include: The creation module is used to create the current backup log file and establish a hard link between the current backup log file and the current source log file; The synchronous write module is used to continuously write log content to the current source log file and synchronously write the log content to the current backup log file through the current hard link; The judgment module is used to determine whether the log content already written in the current source log file has met the preset conditions; if so, the creation module and the synchronous writing module are used to write the subsequent log content to the next source log file. The creation module is also used for: creating the next backup log file; deleting the current source log file to remove the current hard link; and establishing a next hard link between the next source log file and the next backup log file. The synchronous writing module is also used to: continuously write subsequent log content to the next source log file, and synchronize the written log content to the next backup log file through the next hard link.

7. The apparatus as claimed in claim 6, characterized in that, The system is used to automatically generate the next source log file when the current source log file is deleted.

8. The apparatus as claimed in claim 6, characterized in that, The system is used to generate a first source log file and a second source log file; When the first source log file is deleted as the current source log file, the second source log file is used as the next source log file.

9. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method as described in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, performs the method as described in any one of claims 1-5.

Images