Log writing method, device, electronic device, and storage medium

Abstract

A log writing method, device, electronic device and storage medium. The method includes: obtaining the file operation of the business program (S110); obtaining a first interface corresponding to the file operation from multiple system interfaces configured by the log file system, wherein the system interface is an interface defined based on the portable operating system interface standard (S120); obtaining a second interface corresponding to the first interface from multiple interfaces configured by the back-end log storage system (S130); calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system (S140). Through the above method, the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the writing and reading performance of the local disk, and high-speed writing of the log is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of computer data storage, and more specifically, to a log writing method, device, electronic device, and storage medium. Background Art

[0002] Log files, or collections of files used to record system operational events, are becoming increasingly important and relied upon. Retrieving log files can solve numerous program operation issues, and unified collection and analysis of log files allows for in-depth analysis of program behavior. However, related log writing methods are limited by the write and read performance of local disks. Summary of the Invention

[0003] In view of the above problems, the present application proposes a log writing method, device, electronic device and storage medium to improve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a log writing method, the method comprising: obtaining a file operation of a business program; obtaining a first interface corresponding to the file operation from a plurality of system interfaces configured in a log file system, wherein the system interface is an interface created based on a portable operating system interface standard; obtaining a second interface corresponding to the first interface from a plurality of interfaces configured in a back-end log storage system; and calling the second interface to write the log file corresponding to the file operation in the memory into the back-end log storage system.

[0005] In the second aspect, an embodiment of the present application provides a log writing device, which includes: an operation acquisition unit for acquiring file operations of business programs; a first interface acquisition unit for acquiring a first interface corresponding to the file operation from multiple system interfaces configured in the log file system, wherein the system interface is an interface created based on the portable operating system interface standard; a second interface acquisition unit for acquiring a second interface corresponding to the first interface from multiple interfaces configured in the back-end log storage system; and a writing unit for calling the second interface to write the log file corresponding to the file operation in the memory into the back-end log storage system.

[0006] In a third aspect, an embodiment of the present application provides an electronic device comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which program code is stored, wherein the above method is executed when the program code is run.

[0008] The embodiments of the present application provide a method, device, electronic device and storage medium for log writing. Obtain the file operation of the business program, obtain a first interface corresponding to the file operation from a plurality of system interfaces configured by the log file system, wherein the system interface is an interface defined based on the portable operating system interface standard, obtain a second interface corresponding to the first interface from a plurality of interfaces configured by the back-end log storage system, and call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system. Through the above method, the generated log file can be directly written into the memory first, and then the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory into the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the writing and reading performance of the local disk, thereby achieving high-speed writing of the log. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 The following is a diagram showing the architecture of a log writing system proposed in this application;

[0011] Figure 2 A flow chart of a log writing method proposed in one embodiment of the present application is shown;

[0012] Figure 3 A schematic diagram of a scenario for calling an interface proposed in an embodiment of the present application is shown;

[0013] Figure 4 A flow chart of a log writing method proposed in another embodiment of the present application is shown;

[0014] Figure 5 A flow chart of a log writing method proposed in another embodiment of the present application is shown;

[0015] Figure 6 A flow chart of a log writing method proposed in another embodiment of the present application is shown;

[0016] Figure 7 A structural block diagram of a log writing device proposed in an embodiment of the present application is shown;

[0017] Figure 8A structural block diagram of a writing unit proposed in an embodiment of the present application is shown;

[0018] Figure 9 A structural block diagram of a log writing device proposed in an embodiment of the present application is shown;

[0019] Figure 10 A structural block diagram of an electronic device for executing a log writing method according to an embodiment of the present application is shown in real time in the present application;

[0020] Figure 11 The present invention shows a storage unit in real time for storing or carrying program codes for implementing the log writing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] Log files are recording files or file collections used to record system operation events. They are becoming increasingly important and relied upon. By retrieving logs, many program operation problems can be solved. By uniformly collecting and analyzing logs, in-depth analysis of program operation behavior can be performed.

[0023] However, the inventors found in their research on related log writing methods that these methods have high requirements for the format and path of the logs, requiring the log path to be configured in advance in the configuration file of the collection end, which is very inflexible. In addition, the real-time performance of the related log writing methods is also relatively poor. In order to improve the collection performance, a higher aggregation time and number of entries are configured, and the logs are generally written to the back-end storage system every 5 seconds or 1,000 logs. The related log writing methods are all secondary bypass reads, which are not only limited in performance by the write and read performance of the local disk, but also increase the load pressure on the application's log hard disk.

[0024] Therefore, the inventor proposed the method in this application of obtaining the file operation of the business program, obtaining a first interface corresponding to the file operation from multiple system interfaces configured by the log file system, wherein the system interface is an interface created based on the portable operating system interface standard, obtaining a second interface corresponding to the first interface from multiple interfaces configured by the back-end log storage system, calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system, and the log file system calls the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system without writing to the actual disk. Log writing is no longer limited by the write and read performance of the local disk, thereby realizing a method, device, electronic device and storage medium for high-speed log writing.

[0025] See also Figure 1 , an embodiment of the present application provides a log writing system 100, and the system 100 may include a log service end 110 and a log storage end 120. The log service end 110 may include a log file system 112 and a first interface module 114, and the log storage end 120 may include a back-end log storage system 122 and a second interface module 124. Furthermore, the first interface module 114 is configured with multiple system interfaces, and the second interface module 124 is configured with multiple interfaces corresponding to the system interfaces. The log file system 112 can communicate with the back-end log storage system 122 through the first interface module 114 and the second interface module 124.

[0026] See also Figure 2 , an embodiment of the present application provides a log writing method, which is applied to a log writing system, and the method includes:

[0027] Step S110: Obtain the file operation of the business program.

[0028] As a method, the file operation may include an open file operation, a read file operation, a write file operation, a rename operation, a delete operation, and a close file operation. The log file system obtains the current file operation of the business program, such as opening a file or reading and writing a file.

[0029] Step S120: Acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface defined based on a portable operating system interface standard.

[0030] As a method, the log file system is a distributed log file system. The log file system is configured with multiple system interfaces, which are interfaces related to the log file system based on the Portable Operating System Interface of Unix (POSIX) standard. The Portable Operating System Interface standard related to the log file system defines a set of source code level interfaces to enable business programs to operate files in a consistent manner. A log file system that supports the Portable Operating System Interface provides the implementation of corresponding functions and semantics corresponding to all POSIX file system interfaces. The Portable Operating System Interface defines multiple file operations that can operate and control all aspects of files. The multiple system interfaces provided by the log file system may include system interfaces such as open, read, write, rename, symlink, rm, and close. Among them, the Portable Operating System Interface standard describes the operating system's call service interface, which is used to ensure that the compiled business programs can be ported and run on multiple operating systems at the source code level.

[0031] Specifically, the log file system obtains a system interface corresponding to the file operation of the business program from multiple system interfaces configured in the system.

[0032] Step S130: Acquire a second interface corresponding to the first interface from multiple interfaces configured in the backend log storage system.

[0033] As a method, multiple interfaces are pre-configured in the back-end log storage system, and the multiple interfaces correspond one-to-one to multiple system interfaces in the log file system. The log file system can obtain the interface corresponding to the system interface from the multiple interfaces pre-configured in the back-end log storage system based on the system interface corresponding to the file operation corresponding to the business program. For example, Figure 3 As shown, the first interface module is configured with interface A, interface B and interface C, and the second interface module is configured with interface D, interface E and interface F. If the system interface corresponding to the file operation of the business program is interface A, the log file system can find the interface corresponding to interface A from the multiple interfaces configured in the second interface module. If it is interface E, the log file system can call interface E to write the log file corresponding to the file operation of the business program stored in the memory into the back-end log storage system.

[0034] Step S140: calling the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system.

[0035] As one approach, the log file system calls the second interface obtained through the above operation to write the log file generated in the memory of the log file system and corresponding to the file operation of the business program into the backend log storage system.

[0036] An embodiment of the present application provides a log writing method, which obtains the file operation of a business program, obtains a first interface corresponding to the file operation from a plurality of system interfaces configured by a log file system, wherein the system interface is an interface defined based on a portable operating system interface standard, obtains a second interface corresponding to the first interface from a plurality of interfaces configured by a back-end log storage system, and calls the second interface to write the log file corresponding to the file operation in the memory into the back-end log storage system. Through the above method, the generated log file can be directly written into the memory first, and then the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory into the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the writing and reading performance of the local disk, thereby achieving high-speed writing of the log.

[0037] See also Figure 4 , an embodiment of the present application provides a log writing method, which is applied to a log writing system, and the method includes:

[0038] Step S210: Obtain the file operation of the business program.

[0039] As a method, the file operations of the business program are obtained from the system kernel of the system where the log file system is located, and the file operations of the business program are obtained by the system kernel from the mount path of the log file system. Wherein, the system where the log file system is located is an operating system, which can optionally be a Linux operating system.

[0040] Specifically, the journaling file system runs on a Linux operating system. The journaling file system is started and mounted to a directory. Business programs operate on files at the mount point of the journaling file system's mount path, such as opening files or reading and writing files. The Linux kernel forwards the business program's file operations to the virtual file system. The virtual file system forwards the business program's file operations to the journaling file system kernel module. The journaling file system kernel module forwards the business program's file operations to the journaling file system user-mode program.

[0041] The role of the Virtual File System (VFS) is to use standard system calls to read and write different file systems located on different physical media, providing a unified operating interface and application programming interface for various file systems. The virtual file system is a glue layer that allows system calls such as open(), read(), and write() to work without considering the underlying storage media and file system type. Each file system at the virtual file system layer consists of a mount point, superblock, directory, and inode; mounting a file system actually creates these four data structures, making them very important and closely related. Because the virtual file system requires the actual file system to provide the above data structures, different file systems can access each other at the virtual file system layer.

[0042] Step S220: Acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface created based on a portable operating system interface standard.

[0043] Step S230: Acquire a second interface corresponding to the first interface from multiple interfaces configured in the backend log storage system.

[0044] Step S240: creating a log file corresponding to the file operation.

[0045] As one approach, the log file is created at the mount point of the mount path of the log file system according to the portable operating system interface standard.

[0046] Optionally, metadata information of the log file is created, wherein the metadata information includes the file name, creation information, file size, and mount path of the log file; and the metadata information is written into a target database in the log file system.

[0047] Metadata, also known as intermediary data or relay data, is data that describes data, primarily information describing data attributes. It can be used to support functions such as indicating storage location, historical data, resource search, and file logging. The creation information can include information such as the creation time and creation path of a log file.

[0048] Furthermore, writing the metadata information into the target database in the log file system may include: obtaining a log identifier of the log file from the backend log storage system according to the mount path; and writing the log identifier into the metadata information.

[0049] The target database is a key / value database. A key-value database is a database that stores data as key-value pairs, similar to a map in Java. The entire database can be thought of as a large map, where each key corresponds to a unique value. The data structure of the key / value database is a hash table.

[0050] Furthermore, the log file system can obtain the log file identifier by acquiring the metadata information of the log file in the target database, and the log data in the log file can be read through the log file identifier. Furthermore, by analyzing the read log data, it can be determined whether there is a problem with the operation of the business program, and if there is a problem, it can be solved in a timely manner. The log file system can pre-set the format of the log file metadata information, which can include the format of the file name, the file size of the log file, and the creation time of the log file. When creating the metadata information of a new log file, the format of the metadata information of the newly created log file can be compared with the format of the metadata information of the pre-set log file to determine whether the format of the metadata information of the newly created log file is incorrect. Exemplarily, the log file system can compare the file name of the newly created log file read with the format of the file name pre-set in the log file system to determine whether the file name format is correct. If the file name format of the log file is incorrect, the file name format of the log file is modified in a timely manner.

[0051] Step S250: Open the log file and write the log data into the log file.

[0052] As a method, a business program may be operated at the mount point of the log file system, open the log file created above according to the portable operating system interface standard, and write the log data generated in the log file system memory into the log file.

[0053] Step S260: writing the log file into the backend log storage system through the second interface.

[0054] As a method, log files corresponding to file operations of business programs can be called from multiple interfaces of the backend log storage system to write into the backend log storage system.

[0055] The embodiment of the present application provides a log writing method, which obtains the file operation of the business program, obtains a first interface corresponding to the file operation from multiple system interfaces configured by the log file system, wherein the system interface is an interface defined based on the portable operating system interface standard, obtains a second interface corresponding to the first interface from multiple interfaces configured by the back-end log storage system, creates a log file corresponding to the file operation, opens the log file, writes the log data to the log file, and writes the log file to the back-end log storage system through the second interface. Through the above method, the generated log file can be directly written to the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the write and read performance of the local disk, and the high-speed writing of the log is achieved. Furthermore, when creating the metadata information of the log file, the metadata information of the newly created log file is compared with the metadata information of the log file pre-set by the file system, and the metadata information with errors is modified in time to ensure the correctness of the metadata information.

[0056] See also Figure 5 , an embodiment of the present application provides a log writing method, which is applied to a log writing system, and the method includes:

[0057] Step S310: Initialize the log file system.

[0058] As a way, the initialization of the log file system can be understood as initializing the log file system. The steps of initializing the log file system may include: creating a metadata instance of the target database to determine the location of the log file in the log file system according to the metadata instance; creating a cache data instance of the target database to access the log data of the log file according to the cache data instance; initializing the root directory of the log file to initialize the log file system.

[0059] Specifically, the metadata instance is log file system metadata, which can be used to locate the location of the log file in the log file system. The cache data instance can be understood as a handle to open the log file, and the business program can access the log data in the log file through the cache data instance. The root directory is created when the log file system is created, and its purpose is to store directory entries of subdirectories (also called folders) or files.

[0060] Step S320: Obtain the file operation of the business program.

[0061] Step S330: Acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface created based on a portable operating system interface standard.

[0062] Step S340: Acquire a second interface corresponding to the first interface from multiple interfaces configured in the backend log storage system.

[0063] Step S350: Obtain a target sub-thread from multiple sub-threads.

[0064] As one embodiment, the step of obtaining a target sub-thread from a plurality of sub-threads includes: obtaining a plurality of sub-threads of the business program; and obtaining a sub-thread corresponding to a file operation of the business program from the plurality of sub-threads as the target sub-thread.

[0065] Specifically, multiple sub-threads can be created according to the file operations of the business program, and different sub-threads can be used to perform different file operations. Furthermore, the corresponding sub-thread can be determined according to the current file operation of the business program, and the corresponding sub-thread can be used as the target sub-thread to execute the subsequent call to the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.

[0066] Step S360: Based on the target child thread, the second interface is called to write the log file corresponding to the file operation in the memory into the back-end log storage system; wherein, the writing speed of the log file corresponding to the file operation in the memory written by the target child thread to the back-end log storage system is controlled by the token bucket algorithm.

[0067] As a method, the step of calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system includes: obtaining an interface call request for calling the second interface, the interface call request carrying an encrypted signature; performing signature verification on the interface call request; if the signature verification passes, calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system; if the signature verification fails, stopping calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.

[0068] Specifically, to prevent malicious tampering during API calls, API calls require signatures. The backend log storage system will verify the signatures based on the request parameters, and requests with inconsistent signatures will be rejected.

[0069] Furthermore, the request parameter may be an identifier corresponding to the interface, or a type identifier of a business program calling the interface, etc. The log system program may verify whether the identifier corresponding to the interface is correct, obtain verification whether the type identifier of the business program calling the interface is consistent, etc. to use the second interface.

[0070] Step S370: Return the first interface to the business program according to the mount path, so that the business program can determine whether the file operation corresponding to the first interface is completed.

[0071] As a method, after calling the second interface based on the target child thread to write the log file corresponding to the file operation in the memory into the back-end log storage system, the first interface corresponding to the second interface is returned to the business program according to the mount path of the log file system. The business program can judge whether the file operation corresponding to the first interface of the business program is completed based on whether the first interface is received. If the first interface is received, it is determined that the file operation corresponding to the first interface is completed. If the first interface is not received, it is determined that the file operation corresponding to the first interface is not completed.

[0072] The embodiment of the present application provides a log writing method, which obtains the file operation of the business program, obtains the first interface corresponding to the file operation from the multiple system interfaces configured by the log file system, wherein the system interface is an interface defined based on the portable operating system interface standard, obtains the second interface corresponding to the first interface from the multiple interfaces configured by the back-end log storage system, obtains the target sub-thread from the multiple sub-threads, and calls the second interface based on the target sub-thread to write the log file corresponding to the file operation in the memory into the back-end log storage system. Through the above method, the generated log file can be directly written to the memory first, so that the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the write and read performance of the local disk, and the high-speed writing of the log is achieved. Further, when calling the second interface, first verify whether the call request is legal. If it is legal, call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system, thereby improving the security of the interface call process.

[0073] See also Figure 6 , an embodiment of the present application provides a log writing method, which is applied to a log writing system, and the method includes:

[0074] Step S410: Obtain a target sub-thread from multiple sub-threads.

[0075] As a way, when starting the log file system, the system can automatically create a main thread of the log file system. Then, multiple sub-threads can be created through the main thread. Furthermore, tasks can be assigned to the multiple sub-threads. Different sub-threads can handle different tasks or the same tasks. The processing tasks are generally instructions issued by the system to perform logical processing on specific files. When the system needs to process multiple files, multiple sub-threads will be generated according to the main thread for parallel processing. This parallel processing of multiple sub-threads can speed up the processing speed, but in order to achieve load balancing, the sub-threads created will have a certain limit. The preset number can generally be determined based on parameters such as the amount of data processed or CPU performance.

[0076] Optionally, creating a preset number of sub-threads further includes dividing the processing task and allocating an execution task amount to each sub-thread. For example, if there are 10 files to be processed in a log file system, the log file system may create 5 threads based on the number of files to be processed, with each thread calculating 2 files. In the process of allocating the execution task amount, the processing amount may be distributed equally or unequally.

[0077] Furthermore, the log file system may obtain a sub-thread corresponding to the current file operation of the business program from the multiple sub-threads determined by the above operations, and use the sub-thread as the target sub-thread.

[0078] Furthermore, the multiple sub-threads can be named and stored in a target database. The task progress of the file operations performed by the multiple sub-threads can also be stored in real time. A mapping relationship between the sub-threads and the task progress can be established, and the corresponding task progress can be obtained by searching for the sub-thread names. For example, the name of the target sub-thread can be obtained, and the task progress of the file operation corresponding to the sub-thread name can be searched in the target database.

[0079] Step S420: Acquire the file operation of the business program based on the target sub-thread.

[0080] As one way, the current file operation of the business program is obtained based on the mount point of the mount path of the log file system of the target sub-thread obtained by the above method.

[0081] Step S430: Acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system based on the target sub-thread.

[0082] As a method, before obtaining the first interface corresponding to the file operation from multiple system interfaces configured in the log file system based on the target sub-thread, the log file system user-state program obtains the file operation of the business program from the mount path of the log file system based on the target sub-thread.

[0083] Step S440: Based on the target sub-thread, a second interface corresponding to the first interface is obtained from a plurality of interfaces configured in the backend log storage system.

[0084] Step S450: Based on the target child thread, the second interface is called to write the log file corresponding to the file operation in the memory into the back-end log storage system; wherein, the writing speed of the log file corresponding to the file operation in the memory written by the target child thread to the back-end log storage system is controlled by the token bucket algorithm.

[0085] As a way to prevent network congestion when transmitting data in the network, it is necessary to limit the flow of traffic out of the network so that the traffic is sent out at a relatively uniform speed. The token bucket algorithm is the most commonly used algorithm in network traffic shaping (Traffic Shaping) and rate limiting (Rate Limiting). Typically, the token bucket algorithm can be used to control the amount of data sent to the network and allow bursts of data to be sent. Therefore, in order to ensure the stability of the log file system and the back-end log storage system, the token bucket algorithm can be used to control the write speed of the target child thread to write the log file corresponding to the file operation of the business program in memory to the back-end log storage system.

[0086] Furthermore, multiple sub-threads are restricted by the same token bucket algorithm. In the case of multiple sub-threads, the token bucket algorithm can be used to specify which sub-thread has the permission to write to the log file.

[0087] The embodiment of the present application provides a log writing method, which obtains a target sub-thread from multiple sub-threads, obtains the file operation of the business program based on the target sub-thread, obtains a first interface corresponding to the file operation from multiple system interfaces configured by the log file system based on the target sub-thread, wherein the system interface is an interface defined based on the portable operating system interface standard, obtains a second interface corresponding to the first interface from multiple interfaces configured by the back-end log storage system based on the target sub-thread, and writes the log file corresponding to the file operation in the memory to the back-end log storage system based on the target sub-thread. Through the above method, the generated log file can be directly written to the memory first, and then the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the write and read performance of the local disk, thereby achieving high-speed writing of the log. Furthermore, the writing speed of the log file is limited by the token bucket algorithm, which ensures the stability of the log file system and the back-end log storage system.

[0088] See also Figure 7 , an embodiment of the present application provides a log writing device 500, the device 500 comprising:

[0089] The operation acquisition unit 510 is used to acquire the file operation of the business program.

[0090] The operation acquisition unit 510 is further configured to acquire the file operation of the business program from the system kernel of the system where the log file system is located. The file operation of the business program is acquired by the system kernel from the mount path of the log file system.

[0091] The first interface acquiring unit 520 is configured to acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface created based on a portable operating system interface standard.

[0092] The second interface acquiring unit 530 is configured to acquire a second interface corresponding to the first interface from a plurality of interfaces configured in the backend log storage system.

[0093] The writing unit 540 is configured to call the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system.

[0094] The writing unit 540 is also used to obtain an interface call request for calling the second interface, and the interface call request carries an encrypted signature; perform signature verification on the interface call request; if the signature verification passes, call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system; if the signature verification fails, stop calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system.

[0095] See also Figure 8 , an embodiment of the present application provides a writing unit 540, the writing unit 540 including:

[0096] The file creation module 542 is configured to create a log file corresponding to the file operation.

[0097] The file creation module 542 is further configured to create the log file at the mount point of the mount path of the log file system according to the portable operating system interface standard.

[0098] Furthermore, the file creation module 542 is also used to create metadata information of the log file, wherein the metadata information includes the file name, creation information, file size and mount path of the log file; and write the metadata information into the target database in the log file system.

[0099] Furthermore, the file creation module 542 is further configured to obtain a log identifier of the log file from the backend log storage system according to the mount path; and write the log identifier into the metadata information.

[0100] The data writing module 544 is used to open the log file and write the log data into the log file.

[0101] The log writing module 546 is configured to write the log file into the backend log storage system via the second interface.

[0102] See also Figure 9 , an embodiment of the present application provides a log writing device 600, the device 600 comprising:

[0103] Initialization unit 610 is configured to initialize the log file system. Specifically, it creates a metadata instance of the target database to determine the location of the log file in the log file system based on the metadata instance; creates a cache data instance of the target database to access the log data of the log file based on the cache data instance; and initializes the root directory of the log file to initialize the log file system.

[0104] The operation acquisition unit 620 is configured to acquire the file operation of the business program, specifically, based on the target sub-thread.

[0105] Further, it is used to obtain multiple sub-threads of the business program; and obtain a sub-thread corresponding to the file operation of the business program from the multiple sub-threads as the target sub-thread.

[0106] The first interface acquisition unit 630 is configured to acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface created based on a portable operating system interface standard.

[0107] Specifically, it is used to obtain the first interface corresponding to the file operation from multiple system interfaces configured in the log file system based on the target child thread

[0108] The second interface acquiring unit 640 is configured to acquire a second interface corresponding to the first interface from a plurality of interfaces configured in the backend log storage system.

[0109] Specifically, it is used to obtain a second interface corresponding to the first interface from multiple interfaces configured in the backend log storage system based on the target sub-thread.

[0110] The writing unit 650 is configured to call the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system.

[0111] Specifically, it is used to call the second interface based on the target child thread to write the log file corresponding to the file operation in the memory into the back-end log storage system; wherein, the writing speed of the log file corresponding to the file operation in the memory written by the target child thread to the back-end log storage system is controlled by the token bucket algorithm.

[0112] The judgment unit 660 is configured to return the first interface to the business program according to the mount path, so that the business program can judge whether the file operation corresponding to the first interface is completed.

[0113] It should be noted that the device embodiment in this application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0114] The following will be combined Figure 10 An electronic device provided by this application is described.

[0115] See also Figure 10Based on the above-mentioned log writing method and apparatus, the embodiments of the present application also provide another electronic device 200 that can execute the above-mentioned log writing method. The electronic device 200 includes one or more (only one is shown in the figure) processors 102, a memory 104, and a network module 106 that are coupled to each other. Among them, the memory 104 stores a program that can execute the content of the above-mentioned embodiments, and the processor 102 can execute the program stored in the memory 104.

[0116] The processor 102 may include one or more processing cores. The processor 102 utilizes various interfaces and circuits to connect various components within the electronic device 200. It executes instructions, programs, code sets, or instruction sets stored in the memory 104, and accesses data stored in the memory 104 to perform various functions and process data within the electronic device 100. Optionally, the processor 102 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 102 and may be implemented separately via a communication chip.

[0117] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the terminal 100 during use (such as a phone book, audio and video data, chat history data), etc.

[0118] The network module 106 is used to receive and transmit electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices, such as communicating with an audio playback device. The network module 106 may include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a subscriber identity module (SIM) card, a memory, etc. The network module 106 can communicate with various networks such as the Internet, an enterprise intranet, a wireless network, or communicate with other devices via a wireless network. The above-mentioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. For example, the network module 106 can exchange information with a base station.

[0119] Please refer to Figure 11 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0120] The computer-readable storage medium 800 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in a suitable form.

[0121] The present application provides a log writing method, device, electronic device and storage medium, which obtains the file operation of a business program, obtains a first interface corresponding to the file operation from multiple system interfaces configured by the log file system, wherein the system interface is an interface defined based on the portable operating system interface standard, obtains a second interface corresponding to the first interface from multiple interfaces configured by the back-end log storage system, and calls the second interface to write the log file corresponding to the file operation in the memory into the back-end log storage system. Through the above method, the generated log file can be directly written into the memory first, and then the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory into the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the writing and reading performance of the local disk, thereby achieving high-speed writing of the log.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A log writing method, characterized in that: The method comprises: Obtain file operations for business procedures; Acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface defined based on a portable operating system interface standard; Acquire a second interface corresponding to the first interface from a plurality of interfaces configured in the backend log storage system; Calling the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system; The calling of the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system includes: Get the target child thread from multiple child threads; Calling the second interface based on the target child thread to write the log file corresponding to the file operation in the memory into the backend log storage system; The token bucket algorithm is used to control the writing speed of the target sub-thread in writing the log file corresponding to the file operation in the memory into the back-end log storage system.

2. The method according to claim 1, characterized in that The file operation of obtaining the business program includes: The file operation of the business program is obtained from the system kernel of the system where the log file system is located. The file operation of the business program is obtained by the system kernel from the mount path of the log file system.

3. The method according to claim 1, characterized in that The calling of the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system includes: Creating a log file corresponding to the file operation; Open the log file and write the log data into the log file; The log file is written into the backend log storage system through the second interface.

4. The method according to claim 3, characterized in that The creating of a log file corresponding to the file operation includes: The log file is created at the mount point of the mount path of the log file system according to the portable operating system interface standard.

5. The method according to claim 1, characterized in that: The method further comprises: Creating metadata information of the log file, wherein the metadata information includes the file name, creation information, file size, and mount path of the log file; The metadata information is written into a target database in the log file system.

6. The method according to claim 5, characterized in that Writing the metadata information into the target database includes: Obtaining the log identifier of the log file from the backend log storage system according to the mount path; The log identifier is written into the metadata information.

7. The method according to claim 6, characterized in that The file operation of obtaining the business program also includes: Initializing the log file system; Initializing the log file system includes: Creating a metadata instance of the target database to determine a location of the log file in the log file system according to the metadata instance; Creating a cache data instance of the target database to access the log data of the log file according to the cache data instance; Initialize the root directory of the log file to initialize the log file system.

8. The method according to any one of claims 2 to 7, characterized in that: After calling the second interface to write the log file corresponding to the file operation in the memory into the back-end log storage system, the method further includes: The first interface is returned to the business program according to the mount path, so that the business program can determine whether the file operation corresponding to the first interface is completed.

9. The method according to any one of claims 5 to 7, characterized in that: The target database is a key / value database, and the data structure of the key / value database is a hash table structure.

10. The method according to claim 1, characterized in that Before the file operation of obtaining the business program, the method further includes: Get the target child thread from multiple child threads; The file operation of obtaining the business program includes: Acquire a file operation of a business program based on the target sub-thread; The step of obtaining a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface created based on a portable operating system interface standard, comprises: Acquire, based on the target sub-thread, a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system; The acquiring a second interface corresponding to the first interface from a plurality of interfaces configured in the back-end log storage system includes: Acquire, based on the target sub-thread, a second interface corresponding to the first interface from a plurality of interfaces configured in a backend log storage system; The calling of the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system includes: Calling the second interface based on the target child thread to write the log file corresponding to the file operation in the memory into the backend log storage system; The token bucket algorithm is used to control the writing speed of the target sub-thread in writing the log file corresponding to the file operation in the memory into the back-end log storage system.

11. The method according to claim 1 or 10, characterized in that The acquiring of a target sub-thread from a plurality of sub-threads includes: Acquire multiple sub-threads of the business program; A sub-thread corresponding to the file operation of the business program is obtained from the multiple sub-threads as the target sub-thread.

12. The method according to claim 1, characterized in that The calling of the second interface to write the log file corresponding to the file operation in the memory into the backend log storage system includes: Obtaining an interface call request for calling the second interface, where the interface call request carries an encrypted signature; Performing signature verification on the interface call request; If the signature verification passes, the second interface is called to write the log file corresponding to the file operation in the memory into the backend log storage system.

13. The method according to claim 12, characterized in that The method further comprises: If the signature verification fails, the calling of the second interface is stopped to write the log file corresponding to the file operation in the memory into the back-end log storage system.

14. The method according to claim 1, wherein The log file system is a distributed log file system.

15. A log writing device, characterized in that: The device comprises: An operation acquisition unit, used to acquire file operations of business programs; A first interface acquiring unit, configured to acquire a first interface corresponding to the file operation from a plurality of system interfaces configured in the log file system, wherein the system interface is an interface defined based on a portable operating system interface standard; A second interface acquiring unit, configured to acquire a second interface corresponding to the first interface from a plurality of interfaces configured in the backend log storage system; A writing unit is used to call the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system; calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system includes: obtaining a target child thread from multiple child threads; calling the second interface based on the target child thread to write the log file corresponding to the file operation in the memory to the back-end log storage system; wherein, the writing speed of the target child thread to write the log file corresponding to the file operation in the memory to the back-end log storage system is controlled by a token bucket algorithm.

16. The device according to claim 15, characterized in that The operation acquisition unit is further configured to acquire file operations of business programs from a system kernel of a system where the log file system is located. The file operations of business programs are acquired by the system kernel from a mount path of the log file system.

17. The device according to claim 15, characterized in that The writing unit includes: A file creation module, used to create a log file corresponding to the file operation; A data writing module is used to open the log file and write log data into the log file; A log writing module is used to write the log file into the back-end log storage system through the second interface.

18. An electronic device, characterized in that: The method comprises one or more processors and a memory; one or more programs are stored in the memory and are configured to execute the method according to any one of claims 1 to 14 by the one or more processors.

19. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 14 is executed.

Images