High-performance log recording method and device for cloud base station system

Abstract

The embodiment of the invention discloses a high-performance log recording method and device for a cloud base station system, and the method comprises the steps: a recording step: distributing a shared memory for each business process of the cloud base station system, and storing a log recorded by each business process in the corresponding shared memory, the log comprises a file name address, a file name line number, log filtering information, a timestamp, a log character string address and a log parameter; and a storage step: periodically polling whether cached logs exist in each shared memory, if so, reading all the logs cached in the shared memory, and writing the logs into a local file in batches. According to the method, the log recording efficiency is improved, more logs can be recorded under the condition that the performance is influenced to the minimum extent, and the maintenance and measurement requirements of a service side are met.

Description

Technical Field

[0001] The present invention relates to the field of base station technology, and in particular to a high-performance log recording method and device for a cloud base station system. Background Art

[0002] Traditional log records cannot meet the log requirements of base station systems, mainly in the following aspects: (1) In the 5G base station system, some services have very high real-time performance. For example, in a 4T4R cell with a 30KHz subcarrier spacing and 100M bandwidth, the interval of a slot is 500us. Usually, due to system scheduling, the actual time is even shorter, so the service processing time is very tight. However, in some cases, because maintenance and testing have to add some logs to locate problems or check status, the time of each log record must be controllable and as short as possible to minimize the impact on the service. Therefore, the real-time performance of the log is very high. Traditional logging uses system calls such as printf, thread locks, and semaphores, which makes real-time performance uncontrollable.

[0003] (2) The business traffic of 5G is very large. For example, the peak traffic of the above specifications exceeds 3 Gbps in downlink and 1 Gbps in uplink. If you want to log the status or statistics of some data in fapi processing, pdcp, etc., the log volume is very large, which has the impact of increasing the demand for storage space and causing more system io operations, seriously affecting the overall performance of the system.

[0004] (3) Cloud base stations are deployed on the cloud, and usually many other devices are deployed on the cloud. If too many logs are written, the performance of other devices will be affected, thus affecting the overall performance of the cloud. Summary of the invention

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a high-performance log recording method and device for a cloud base station system, so as to record more logs with minimal impact on performance and meet the maintenance and testing requirements of the business side.

[0006] In order to solve the above technical problems, an embodiment of the present invention proposes a high-performance logging method for a cloud base station system, including: Recording step: Allocate a shared memory for each business process of the cloud base station system, and save the log recorded by each business process each time into the corresponding shared memory, wherein the log includes the file name address, file name line number, log filtering information, timestamp, log string address, and log parameters; Storage steps: Regularly poll each shared memory to see if there are any cached logs. If so, read all the logs cached in the shared memory and write the logs in batches to local files.

[0007] Accordingly, an embodiment of the present invention further provides a high-performance log recording device for a cloud base station system, including a business process end and a background process end. The business process end allocates a shared memory for each business process of the cloud base station system, and saves the log recorded by each business process each time into the corresponding shared memory. The log includes a file name address, a file name line number, log filtering information, a timestamp, a log string address, and log parameters. The background process periodically polls each shared memory to see if there are any cached logs. If so, it reads all the cached logs in the shared memory and writes the logs in batches to local files.

[0008] The beneficial effects of the present invention are as follows: the present invention has no system call when recording logs, each log takes about 100 cycles, about 0.05us, and has very high performance, reaching 10 million pps level per second, which can fully meet the real-time requirements of the business; each log is 64 bytes, and if 1G byte of space is given to the log, 16777216 logs can be stored. If compression is combined, the performance is more impressive, which fully meets the performance requirements of the base station. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is an overall block diagram of a high-performance log recording device for a cloud base station system according to an embodiment of the present invention.

[0010] Figure 2 4 is a flow chart of log recording according to an embodiment of the present invention.

[0011] Figure 3 Schematic diagram of a read / write pointer according to an embodiment of the present invention.

[0012] Figure 4 It is a schematic diagram of the structure of the elf file of an embodiment of the present invention.

[0013] Figure 5 The following is a flow chart of log storage according to an embodiment of the present invention.

[0014] Figure 6 4 is a flow chart of log parsing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention is further described in detail below in conjunction with the drawings and specific embodiments.

[0016] In the embodiments of the present invention, if there are directional indications (such as up, down, left, right, front, back, etc.), they are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0017] In addition, in the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0018] The high-performance logging method for a cloud base station system according to an embodiment of the present invention includes a logging step and a storage step. The logging process of the logging step and the storage step is as follows: Figure 2 shown.

[0019] Recording step: Allocate a shared memory for each business process of the cloud base station system, and save the logs recorded by each business process to the corresponding shared memory. The business process of the present invention does not use any system call when recording logs, ensuring that the system kernel state will not be entered due to recording logs, thereby ensuring real-time performance.

[0020] In the recording step, the shared memory is allocated by each cpu of the cloud base station system. Each business process is first bound to a cpu, and then the cpu allocates the shared memory. The shared memory allocated by each cpu is a fixed-size circular queue. For the business process, each thread is required to be bound to the cpu, and each cpu corresponds to a shared memory. The memory can only be written by the cpu, and the read can only be read by the cpu of the process where rtlogapp is located, usually 0 cores. In this way, multi-process and multi-core influence each other can be avoided as much as possible. At the same time, for the competition problem, only the consistency during reading and writing needs to be considered. If multiple threads are bound to the same cpu, the log recorded by the cpu contains the logs of several threads at the same time. The present invention saves the log recorded by the business process each time only to the shared memory, which is very fast. The shared memory is divided according to the number of cores of the cpu. The business process is bound to which core, and the corresponding shared memory is used to cache the log. This division by core is to facilitate the subsequent log collection.

[0021] The shared memory allocated to each CPU is a circular queue of fixed size. The CPU to which it belongs is responsible for maintaining the write pointer, and the CPU to which rtlogapp belongs is responsible for maintaining the read pointer. Both sides can operate in parallel. Once the read pointer is equal to the write pointer, the reading stops. For the change of the write pointer, the CAS (Compare-and-swap) instruction needs to be used for atomic operation to avoid conflicts when multiple threads operate simultaneously (such as Figure 3As shown in the figure). This method avoids the use of locks and greatly improves the efficiency of logging. For read pointers and some internal shared variable maintenance, atomic variables are used for operations, which is also very efficient.

[0022] The log of the present invention includes the file name address, file name line number, log filtering information, timestamp, log string address, and log parameters, which can minimize the size of the log memory occupied. Each log only records the address of the log string and related parameters, so that a log can be basically controlled within 64 bytes (within a cacheline), thereby reducing the storage space occupied by the overall log.

[0023] The content of each log record in the present invention is controlled within 64 bytes, which is just within a cache line. These 64 bytes need to be used reasonably. The saved content includes: 1. File name address (8 bytes); 2. File name line number (2 bytes); 3. Log filtering information (6 bytes); 4. Timestamp (8 bytes); 5. Log string address (8 bytes); 6. Log parameters (32 bytes).

[0024] Among them, 1, 2, and 4 are automatically filled parameters, and 3, 5, and 6 are filter parameters, strings, and parameters recorded when the business process is used. It can be seen that each log can be controlled within 64 bytes. Compared with traditional logs of hundreds of bytes, the present invention greatly reduces the time to access the memory.

[0025] The log string of the present invention only records one address. The business processes are all elf files, and the strings are stored in the .rodata section of the elf file (such as Figure 4 As shown in the figure), once the process is compiled, the address of the rodata area of ​​the process that is finally loaded into the memory is determined. Therefore, when storing in the present invention, only one address is stored in the shared memory, and the address of the string in the rodata can be reversely deduced from this address. The flow chart of log recording is shown in the figure. Figure 2 shown.

[0026] Storage step: Regularly poll each shared memory to see if there is a cached log. If so, read all the logs cached in the shared memory and write the logs in batches to local files. The polling interval can be set according to the shared memory size, preferably 10us. After the logs cached in the shared memory are written to the local file, the logs originally cached in the shared memory are deleted. The collection of the logs of the present invention is performed by a unified background process rtlogapp, which is responsible for unified handling of the logs of all shared memories, which can be stored in the cloud, can be viewed online, or sent to the remote end via the network.

[0027] Log storage is mainly completed by the background process rtlogapp. rtlogapp will regularly poll the shared memory corresponding to the configured cpu. Each shared memory is configured with 5M bytes, which can cache 5*1024*1024 / 64=81920 logs. Therefore, the interval time is set to 10us, which is usually sufficient. If it overflows, the log will be discarded. Rtlogapp collects logs from a shared memory each time. Instead of writing one log at a time, it puts it into the buffer first and then writes it in batches. For example, if a log is 64 bytes, first copy the 64 bytes to the continuous buffer in the memory, and then write it in batches by sector. This can minimize the number of system IOs, such as Figure 5 shown.

[0028] The log parsing part is completed by the mapping step and the parsing step. Please refer to the detailed process Figure 6 .

[0029] Mapping step: Load the .rodata segment of each business process and store the corresponding string, address, and offset information into a memory mapping table.

[0030] Parsing steps: reversely deduce the address of the corresponding string in the business process based on the log string address in the log file in the local file, find the corresponding string based on the address of the string, and store the corresponding string as string plaintext or directly printf output.

[0031] Log parsing is mainly offline log file parsing. As mentioned above, when storing logs, because the string only stores the address, the log cannot be browsed directly and needs to be parsed offline. When the rtlogapp program starts the process, it will load the .rodata segment of other business process elf and store the corresponding string, address, offset and other information into a memory mapping table. Therefore, when parsing offline, you only need to subtract the corresponding offset from the address in the log file, and then you can find the corresponding string in the table, and then store it as string plain text or directly printf output.

[0032] Please refer to Figure 1The high-performance log recording device of the cloud base station system of the embodiment of the present invention includes a business process end and a background process end. The business process end is composed of multiple business processes, and the background process end is composed of the background process rtlogapp.

[0033] The business process end allocates a shared memory for each business process of the cloud base station system, and saves the log recorded by each business process each time to the corresponding shared memory. The log includes the file name address, file name line number, log filtering information, timestamp, log string address, and log parameters.

[0034] The background process periodically polls each shared memory to see if there are any cached logs. If so, it reads all the cached logs in the shared memory and writes the logs in batches to local files.

[0035] As an implementation method, when the background process is started, the .rodata segment of each business process is loaded and the corresponding string, address and offset information are stored in a memory mapping table.

[0036] The background process reversely deduce the address of the corresponding string in the business process based on the log string address in the log file in the local file, find the corresponding string according to the address of the string, and store the corresponding string as string plain text or directly printf output.

[0037] As an implementation method, the shared memory is a circular queue of a fixed size, the business process end is responsible for the maintenance of the write pointer, and the background process end is responsible for the maintenance of the read pointer.

[0038] As an implementation method, when the business process side and the background process side operate in parallel, if the read pointer is equal to the write pointer, reading is stopped; when the write pointer changes, the background process side uses the CAS instruction to perform atomic operations; when the read pointer and some internally shared variables are maintained, the business process side uses atomic variables for operations.

[0039] The present invention can be applied not only to cloud base station business scenarios, but also to any scenarios with high real-time requirements and high log performance requirements. The technical solution of the present invention is very general, has no coupling relationship with the application layer, and is very convenient for transplantation.

[0040] In addition, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0041] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-performance logging method for a cloud base station system, characterized in that: include: Recording step: Allocate a shared memory for each business process of the cloud base station system, and save the log recorded by each business process each time into the corresponding shared memory, wherein the log includes the file name address, file name line number, log filtering information, timestamp, log string address, and log parameters; Storage steps: Regularly poll each shared memory to see if there are any cached logs. If so, read all the logs cached in the shared memory and write the logs in batches to local files.

2. The high-performance logging method for a cloud base station system according to claim 1, characterized in that: The storage step is followed by the parsing step: According to the log string address in the log file in the local file, the address of the corresponding string in the business process is reversely deduced, and the corresponding string is found according to the address of the string, and the corresponding string is stored as string plain text or directly output by printf.

3. The high-performance logging method for a cloud base station system according to claim 2, characterized in that: The parsing step also includes a mapping step before it: Load the .rodata segment of each business process and store the corresponding string, address, and offset information into a memory mapping table.

4. The high-performance logging method for a cloud base station system according to claim 1, characterized in that: In the recording step, the shared memory is allocated by each CPU of the cloud base station system. Each business process is first bound to a CPU, and then the CPU allocates the shared memory. The shared memory allocated to each CPU is a circular queue of a fixed size.

5. The high-performance logging method for a cloud base station system according to claim 4, characterized in that: The maintenance of the shared memory write pointer is the responsibility of the CPU to which it belongs. For changes to the write pointer, an atomic operation using the CAS instruction is required to avoid conflicts when multiple threads are operating simultaneously.

6. A high-performance log recording device for a cloud base station system, comprising a business process end and a background process end, characterized in that: The business process end allocates a shared memory for each business process of the cloud base station system, and saves the log recorded by each business process each time to the corresponding shared memory. The log includes the file name address, file name line number, log filtering information, timestamp, log string address, and log parameters; The background process periodically polls each shared memory to see if there are any cached logs. If so, it reads all the cached logs in the shared memory and writes the logs in batches to local files.

7. The high-performance log recording device of the cloud base station system according to claim 6, characterized in that: When the background process starts, it loads the .rodata segment of each business process and stores the corresponding string, address and offset information into a memory mapping table.

8. The high-performance log recording device for a cloud base station system according to claim 7, characterized in that: The background process reversely deduce the address of the corresponding string in the business process based on the log string address in the log file in the local file, find the corresponding string according to the address of the string, and store the corresponding string as string plain text or directly printf output.

9. The high-performance log recording device of the cloud base station system according to claim 6, characterized in that: The shared memory is a circular queue of a fixed size. The business process is responsible for maintaining the write pointer, and the background process is responsible for maintaining the read pointer.

10. The high-performance log recording device of the cloud base station system according to claim 9, characterized in that: When the business process side and the background process side operate in parallel, if the read pointer is equal to the write pointer, reading is stopped; when the write pointer changes, the background process side uses the CAS instruction to perform atomic operations.

Images