A method, device and equipment for improving log data read-write efficiency

Abstract

The application relates to the storage field and discloses a method, device and equipment for improving the reading and writing efficiency of log data. The method comprises the following steps: setting a plurality of cache areas for log data caching; writing log data into a current cache area and judging whether the use capacity of the current cache area reaches a preset threshold and whether the next cache area pointed by the current cache area is occupied; in response to the use capacity of the current cache area reaching the preset threshold and the next cache area pointed by the current cache area not being occupied, writing the log data of the current cache area into a storage medium and setting the next cache area pointed by the current cache area as a new generation log data writing area. The disclosed scheme separates the process of caching log data into a cache area and the process of writing into a storage medium, so that the application layer does not perceive the process of writing log data into a storage medium, the log data can be quickly written, and the efficiency of writing log data into a storage medium is improved.

Description

Technical Field

[0001] This invention relates to the field of storage, and more particularly to a method, apparatus, and device for improving the efficiency of log data reading and writing. Background Technology

[0002] Log data is typically a variable-length string, generally consisting of a log header and log data. Each log entry usually occupies a small amount of storage space, ranging from tens to a hundred bytes, and even less after compression using specific algorithms. In practical applications, devices, including embedded systems, often need to record various log data during operation, such as event logs and audit logs, to facilitate technical personnel in analyzing and locating the device's operational status. This log data is generally stored in non-volatile storage media, such as Flash memory, so that the log data is not lost when the embedded device loses power.

[0003] In existing technologies, when storing log data on storage media, the common practice is to first cache the log data to be written in a segment of memory, and then write it to the storage media when the data volume is large enough, in order to reduce the number of read and write operations. However, in this method, log data cannot be written to the cache memory during the writing process to the storage media, resulting in the loss of newly generated log data or the need to wait for the writing to the storage media to be completed before the newly generated log data can be written to the cache memory. This leads to a decrease in efficiency when reading and writing log data from the storage media. Summary of the Invention

[0004] In view of this, the present invention proposes a method, apparatus, and device for improving log data read and write efficiency. The method utilizes multiple cache areas during the log data writing process. Log data is initially cached in one cache area. When the log data in that cache area reaches a switching condition (e.g., the cache area is full or there is insufficient space to store a single log data entry), the log data in that cache area is written to flash memory. Simultaneously, the cache area being cached is switched, and the newly generated log data is cached in the remaining cache area. This separates the process of caching log data into the cache area from the process of writing it to the storage medium. The application layer does not need to be aware of the log data writing process to the storage medium, enabling fast log data writing. Furthermore, a log information table is established within the storage medium and loaded into memory for retrieval. Log data is read from the storage medium based on the log information table, reducing log data retrieval time and improving read efficiency.

[0005] Based on the above objectives, one aspect of the present invention provides a method for improving log data read and write efficiency, the method comprising the following steps: setting up multiple cache areas for log data caching; writing log data into the current cache area, and determining whether the usage capacity of the current cache area has reached a preset threshold and whether the next cache area pointed to by the current cache area is occupied; in response to the current cache area reaching the preset threshold and the next cache area pointed to by the current cache area not being occupied, writing the log data of the current cache area into a storage medium and setting to write newly generated log data into the next cache area pointed to by the current cache area.

[0006] In some embodiments, the method further includes: in response to the fact that the usage capacity of the current cache does not reach a preset threshold within a preset time period, continuing to write newly generated log data into the current cache.

[0007] In some embodiments, the method further includes: in response to the fact that the usage capacity of the current cache area has not reached the preset threshold after a preset time period, determining whether the next cache area pointed to by the current cache area is occupied;

[0008] In response to the fact that the next cache area pointed to by the current cache area is not occupied, the log data of the current cache area is written to the storage medium and the newly generated log data is set to be written to the next cache area pointed to by the current cache area;

[0009] In response to the next cache area pointed to by the current cache area being occupied, the current cache area is set to wait for the next cache area pointed to by the current cache area to be occupied before being released, and then the log data of the current cache area is written to the storage medium and the newly generated log data is set to be written to the next cache area pointed to by the current cache area.

[0010] In some embodiments, the method further includes: in response to the current cache area's usage capacity reaching a preset threshold and the next cache area pointed to by the current cache area being occupied, setting the current cache area to wait for the next cache area pointed to by the current cache area to be occupied before releasing the current cache area, writing the log data of the current cache area to the storage medium, and setting the newly generated log data to be written to the next cache area pointed to by the current cache area.

[0011] In some embodiments, the method further includes: configuring a flag for marking the state of the cache;

[0012] Determining whether the next cache pointed to by the current cache is occupied includes: in response to the flag of the next cache pointed to by the current cache being not busy, determining that the next cache pointed to by the current cache is not occupied; in response to the flag of the next cache pointed to by the current cache being busy, determining that the next cache pointed to by the current cache is occupied.

[0013] In some embodiments, the method further includes:

[0014] A log information table is established in the storage medium. The log information table includes the time point when the stored log data was created, the level of the log data, and the module requirements to which the log data belongs.

[0015] In response to system initialization, the log information table is read from memory;

[0016] In response to receiving a log read request, the log information table is filtered in memory, and the corresponding log data is read from the storage medium according to the filtering results.

[0017] In some embodiments, the step of filtering the log information table in memory in response to receiving a log read request and reading the corresponding log data from the storage medium according to the filtering results includes: filtering the log data in the log information table based on the creation time, level, and module requirements of the log data.

[0018] In some embodiments, setting up multiple caches for caching log data includes setting up multiple caches based on the capacity of the storage medium and the wait time of each cache.

[0019] In another aspect, the present invention provides an apparatus for improving log data read / write efficiency. The apparatus includes: a first module configured to set up multiple cache areas for log data caching; a second module configured to write log data into the current cache area, and determine whether the usage capacity of the current cache area has reached a preset threshold and whether the next cache area pointed to by the current cache area is occupied; and a third module configured to, in response to the current cache area reaching the preset threshold and the next cache area pointed to by the current cache area not being occupied, write the log data of the current cache area into a storage medium and set to write newly generated log data into the next cache area pointed to by the current cache area.

[0020] In another aspect of the present invention, a computer device is also provided, including at least one processor; and a memory storing computer instructions executable on the processor, the instructions, when executed by the processor, implementing the steps of any of the methods described above.

[0021] In another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements any of the method steps described above.

[0022] The present invention has at least the following beneficial effects: The present invention proposes a method, apparatus, and device for improving the efficiency of log data reading and writing. Specifically, the method for improving log data reading and writing efficiency employs multiple buffers during the process of writing log data to a storage medium, separating the process of caching log data in the buffers from the process of writing it to the storage medium. This ensures that the application layer is unaware of the process of writing log data to the storage medium, enabling fast writing of log data and improving the efficiency of writing log data to the storage medium. Furthermore, during the process of reading log data from the storage medium, a log information table is established in the storage medium. Reading log data based on the log information table reduces the log data retrieval process and improves the efficiency of reading log data. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic diagram illustrating an embodiment of a method for improving log data read / write efficiency provided by the present invention;

[0025] Figure 2 A schematic diagram of another embodiment of a method for improving log data read / write efficiency provided in this application;

[0026] Figure 3 This application provides a schematic diagram illustrating the writing of log data to a buffer.

[0027] Figure 4 This application provides a schematic diagram of reading log data from Flash.

[0028] Figure 5 A diagram illustrating the monitoring of the log monitoring process provided in this application;

[0029] Figure 6 A schematic diagram illustrating an embodiment of a device for improving log data read / write efficiency provided by the present invention;

[0030] Figure 7 A schematic diagram illustrating an embodiment of a computer device provided by the present invention;

[0031] Figure 8 This is a schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0032] The following describes embodiments of the present invention. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various alternative forms.

[0033] Furthermore, it should be noted that all uses of the terms "first" and "second" in the embodiments of this invention are for the purpose of distinguishing two entities or parameters with the same name but different names. Therefore, "first" and "second" are merely for convenience of expression and should not be construed as limiting the embodiments of this invention. Subsequent embodiments will not elaborate on this further. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements may include not only those elements but also elements not expressly listed or inherent to those processes, methods, articles, or apparatuses.

[0034] One or more embodiments of this application will now be described with reference to the accompanying drawings.

[0035] Based on the above objectives, the first aspect of the present invention provides an embodiment of a method for improving log data read and write efficiency. Figure 1 The diagram shown is an embodiment of a method for improving log data read / write efficiency provided by the present invention. Figure 1 As shown, a method for improving log data read / write efficiency according to an embodiment of the present invention includes the following steps:

[0036] S1. Configure multiple cache areas for log data caching;

[0037] S2. Write log data to the current cache area, and determine whether the usage capacity of the current cache area has reached a preset threshold and whether the next cache area pointed to by the current cache area is occupied.

[0038] S3. In response to the current cache area's usage capacity reaching a preset threshold and the next cache area pointed to by the current cache area not being occupied, write the log data of the current cache area to the storage medium and set the newly generated log data to be written to the next cache area pointed to by the current cache area.

[0039] According to several embodiments of this application, the method further includes: in response to the fact that the usage capacity of the current cache area has not reached a preset threshold within a preset time period, continuing to write newly generated log data into the current cache area.

[0040] According to several embodiments of this application, the method further includes:

[0041] In response to the fact that the usage capacity of the current cache area has not reached the preset threshold after a preset time period, it is determined whether the next cache area pointed to by the current cache area is occupied;

[0042] In response to the fact that the next cache area pointed to by the current cache area is not occupied, the log data of the current cache area is written to the storage medium and the newly generated log data is set to be written to the next cache area pointed to by the current cache area;

[0043] In response to the next cache area pointed to by the current cache area being occupied, the current cache area is set to wait for the next cache area pointed to by the current cache area to be occupied before being released, and then the log data of the current cache area is written to the storage medium and the newly generated log data is set to be written to the next cache area pointed to by the current cache area.

[0044] According to several embodiments of this application, the method further includes: in response to the current cache area's usage capacity reaching a preset threshold and the next cache area pointed to by the current cache area being occupied, setting the current cache area to wait for the next cache area pointed to by the current cache area to be occupied before releasing the state, writing the log data of the current cache area to the storage medium, and setting the newly generated log data to be written to the next cache area pointed to by the current cache area.

[0045] According to several embodiments of this application, the method further includes: configuring a flag for marking the state of the cache area;

[0046] Determining whether the next cache pointed to by the current cache is occupied includes: if the flag of the next cache pointed to by the current cache is not busy, it is determined that the next cache pointed to by the current cache is not occupied; if the flag of the next cache pointed to by the current cache is busy, it is determined that the next cache pointed to by the current cache is occupied.

[0047] According to several embodiments of this application, the method further includes:

[0048] A log information table is established in the storage medium. The log information table includes the time point when the stored log data was created, the level of the log data, and the module requirements to which the log data belongs.

[0049] In response to system initialization, the log information table is read from memory;

[0050] In response to receiving a log read request, the system filters the log information table in memory and reads the corresponding log data from the storage medium based on the filtering results.

[0051] According to several embodiments of this application, in response to receiving a log read request, filtering the log information table in memory and reading the corresponding log data from the storage medium according to the filtering results includes: filtering the log data in the log information table based on the creation time point, the level of the log data, and the module to which the log data belongs.

[0052] According to several embodiments of this application, setting up multiple cache areas for caching log data includes setting up multiple cache areas according to the capacity of the storage medium and the waiting time of each cache area.

[0053] The following provides specific steps of another embodiment of a method for improving log data read and write efficiency based on the present invention.

[0054] In this embodiment, in the application scenario of an embedded device, the log data of the embedded device is stored in Flash. It is understood that the application scenario of an embedded device is only one of the applicable scenarios for the method of improving log data read / write efficiency of this application. Other applicable scenarios are also within the scope of the applicable scenarios of the method of improving log data read / write efficiency of this application. Similarly, storing log data in Flash is only one applicable scenario; the method of improving log data read / write efficiency of this application can also store log data in other non-volatile storage media.

[0055] Flash memory's read / write speed is significantly slower than CPU speed, and it typically has a limited number of read / write cycles. Exceeding this limit increases the probability of Flash memory damage. Therefore, when storing log data in Flash, it's common practice to cache the log data in a memory segment before writing it to Flash, reducing the number of read / write cycles. This approach can lead to log data not being written to the cache during the Flash writing process, resulting in the loss of newly generated log data or waiting until the Flash writing is complete before writing it to the cache. Log data is generally a variable-length string, typically consisting of a log header and log data. Each log entry occupies a relatively small amount of storage space, ranging from tens to a hundred bytes, and even less after compression using specific algorithms. Flash read / write operations are generally performed in page units, with page sizes typically being 1KB, 2KB, or 4KB. To modify data on a page, the entire page of log data must be read, modified, and then written back to Flash.

[0056] In this embodiment, two buffers, Buffer1 and Buffer2, are used during the process of writing log data to Flash. The capacities of Buffer1 and Buffer2 are adapted to the Flash capacity and the respective latency of Buffer1 and Buffer2. It is understood that in practical applications, the specific number of buffers used should be adapted to the Flash capacity and the latency of each buffer. In this embodiment, log data is first cached in Buffer1. When the log data in Buffer1 fills a page or the remaining space is insufficient to store a single log data entry, all log data in Buffer1 is written to Flash. At this time, log data generated by the application layer is cached in Buffer2, so that the application layer does not need to be aware of the log data writing to Flash, achieving fast log data writing. During the process of reading log data from Flash, a log information table needs to be maintained for fast log data reading. The log information table contains the time information of the first log entry stored in each Flash page and the number of logs at different levels. When it is necessary to read logs based on time or log level, the log information table can be used to determine which data in Flash should be read. After the system boots up, the log information table in Flash should be read into memory for easy subsequent querying, and the log information table should be modified synchronously when writing log data to Flash.

[0057] The following is the specific implementation process of this implementation. Figure 2 A schematic diagram of another embodiment of a method for improving log data read / write efficiency provided in this application is shown, as follows: Figure 2 As shown, the method for improving log data reading and writing efficiency provided in this application mainly includes: initialization of the log module, log monitoring process and log reading or writing, wherein the initialization of the log module is divided into maintenance of the log information table and dual cache initialization.

[0058] (1) Log Module Initialization: The initialization of the log module prepares for the log monitoring thread and log reading / writing. The log information table is stored in a specified area of ​​Flash (usually the beginning of Flash), with a fixed size of 8 bytes, and serves as a description of other Flash areas. During log module initialization, the log information table is read from the specified area of ​​Flash into memory and maintained by the log monitoring thread. The log information table contains information such as the time of the earliest log in this page, the log initiating module, and the number of logs at different levels. It is stored in a specified location in Flash with a fixed size. The time of the earliest log is recorded using a timestamp, and the log initiating module is recorded using a bitmap. Each bit represents a module, and a bit position of 1 indicates that the page contains log data for a certain module.

[0059] Dual buffering is initialized by creating two buffers for log data during the logging module's initialization and registering both buffers with the log monitoring process. Since the log buffers may be modified simultaneously by both the log monitoring process and the logging API, a mutex lock is needed to ensure data consistency.

[0060] (2) Log read / write process: The external interface of the log module is the log recording API and the log reading API. Here, log read / write is the upper-layer application calling the log recording API and the log reading API to record or read log data. The total size of the log data structure is defined as 128 bytes. This is because the size of a typical Flash memory is an integer Kb, which is divisible by 128 bytes. Also, 112 bytes is sufficient to describe a log entry. The log data structure includes the log header and the log data itself.

[0061] Figure 3 This application provides a schematic diagram of writing log data to a cache, as shown below. Figure 3 As shown, log writing involves first writing log data to a buffer. Once the buffer is full, the log monitoring process writes it to Flash. Log writing is triggered by an API call from the upper-layer application. The API stores log data into the buffer according to the designed rules. This API call method is relatively simple; the API parameters only require the log level, log data, log data length, and the module to which it belongs. The API is `int logWriteToFlash(enum LogLevel level, int module, *logData, logDataLenth)`. By default, log records are first recorded in buffer 1. When the data storage space in buffer 1 is insufficient to store the next log data, the buffer will be actively switched and the log monitoring process will be notified. If another buffer is being used by the log monitoring process at this time, the process will wait for the buffer to become available before switching. The buffer switching is implemented through a buffer selector in the data structure. If `bufferSelector` is `LOG_BUFFER_1`, the first buffer is used; if it is `LOG_BUFFER_2`, the second buffer is used; other values ​​are invalid. The conditions for cache switching can be changed. For example, the cache can be switched when 80% of the cache space is used, and the remaining 20% ​​can continue to be cached as long as the switched cache area is available.

[0062] Figure 4 This application provides a schematic diagram of reading log data from Flash memory, as shown below. Figure 4As shown, log reading is generally divided into full reading and reading under specified conditions. Specified conditions typically include the time point, log level, and the module that initiated the log reading. When reading logs under specified conditions, the system first checks the log information table to see if the desired log is available for that page, and then decides whether to read the data from that page. For example, to read logs with a warning level or higher within the last three days, the system first checks the log information table of the specified page in Flash. If the time and log level meet the requirements, the data is then read from Flash. The log reading process is also handled by the upper-layer application calling an API. The requirements for the time point, log level, and the module to which the log belongs need to be clearly defined. The API is `struct logEntry* logReadFromFlash(enumLogLevel level, int module, u32 time, inttimeFlag, int* logSize)`, where `timeFlag` is used to supplement the meaning of the time point; 0 indicates after the `time` time point, and 1 indicates before the `time` time point. `logSize` is the size of the log data that meets the requirements read from Flash, in bytes. The log level, the module to which the log belongs, and the time point cannot all be invalid values ​​simultaneously.

[0063] The log reading process filters the log information table according to the requirements of the upper-layer application. The time point is a crucial filtering condition. If there are no specific requirements for the time point, all log information matching the log level and module will be read. Therefore, all three (log level, module, and time point) cannot be invalid simultaneously. When there are no requirements for the log level and module, all log information will be read. If no matching log is found, the interface should return null with logSize set to 0. If the return value is not null, it indicates that the interface has read valid log data and allocated memory. The upper-layer application should release this memory at an appropriate time (the starting address of the memory is the return value, and its size is logSize).

[0064] (3) Log monitoring process, Figure 5 A diagram illustrating the monitoring of the log monitoring process provided in this application is shown below. Figure 5 As shown, after the log monitoring process starts, it is responsible for monitoring whether the data in the cache needs to be written to Flash: whether a notification to write to Flash is received or whether a timeout has occurred. If no notification to write the log to Flash is received within the specified time, the monitoring process will actively switch the cache and write the log data in the cache to Flash. The timeout is generally set to 2 seconds, but the specific timeout can be set according to the actual situation.

[0065] When the Buffer1 buffer reaches the switching condition, the logWriteToFlash interface will notify the log monitoring process to write the data to Flash and switch the buffer Buffer2 at the same time to save the newly generated log data in Buffer2. As the operation progresses, the two buffers will switch to each other.

[0066] The buffer used by the log monitoring process needs to be marked as Busy, which means setting the pendingBuffer in the log buffer data structure to the corresponding enumeration value. This allows the system to check whether another buffer is available when switching between buffers. For example, if buffer 1 has met the switching conditions and needs to switch to buffer 2, but the pendingBuffer value of buffer 2 is LOG_BUFFER_2, it means that buffer 2 is being used by the log monitoring process. In this case, buffer 1 should wait for buffer 2 to become available before switching.

[0067] A second aspect of the present invention provides an apparatus for improving log data read and write efficiency. Figure 6 The diagram shown is a schematic representation of an embodiment of a device for improving log data read / write efficiency provided by the present invention. Figure 6 As shown, the device for improving log data read / write efficiency provided by the present invention includes: a first module 011, configured to set up multiple cache areas for log data caching; a second module 012, configured to write log data into the current cache area, and determine whether the usage capacity of the current cache area has reached a preset threshold and whether the next cache area pointed to by the current cache area is occupied; a third module 013, configured to, in response to the current cache area reaching the preset threshold and the next cache area pointed to by the current cache area not being occupied, write the log data of the current cache area into a storage medium and set to write newly generated log data into the next cache area pointed to by the current cache area.

[0068] To achieve the above objectives, a third aspect of the present invention provides a computer device. Figure 7 The diagram shown is a schematic representation of an embodiment of a computer device provided by the present invention. Figure 7 As shown, an embodiment of a computer device provided by the present invention includes the following modules: at least one processor 021; and a memory 022, the memory 022 storing computer instructions 023 that can be executed on the processor 021, the computer instructions 023 implementing the steps of the method described above when executed by the processor 021.

[0069] The present invention also provides a computer-readable storage medium. Figure 8 The diagram shown is a schematic representation of an embodiment of a computer-readable storage medium provided by the present invention. Figure 8As shown, computer-readable storage medium 031 stores a computer program 032 that, when executed by a processor, performs the following steps.

[0070] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for setting system parameters can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0071] Furthermore, the method disclosed in the embodiments of the present invention can also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. When the computer program is executed by the processor, it performs the functions defined in the method disclosed in the embodiments of the present invention.

[0072] Furthermore, the above-described method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to perform the functions of the above-described steps or units.

[0073] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0074] In one or more exemplary designs, functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted via a computer-readable medium. Computer-readable media include computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one location to another. Storage media may be any available medium accessible to a general-purpose or special-purpose computer. By way of example, and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that may be used to carry or store the required program code in the form of instructions or data structures and is accessible to a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. Furthermore, any connection may be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the aforementioned coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are all included in the definition of media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital multifunction discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0075] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0076] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0077] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0078] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0079] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for improving log data read / write efficiency, characterized in that, include: Configure multiple caches for caching log data; After the log monitoring process starts, it is responsible for monitoring whether the data in the cache needs to be written to Flash: whether a notification to write to Flash is received or whether a timeout has occurred. If no notification to write the log to Flash is received within the specified time, the log monitoring process actively switches the cache and writes the log data in the cache to Flash. Write log data to the current cache area, and determine whether the usage capacity of the current cache area has reached a preset threshold and whether the next cache area pointed to by the current cache area is occupied; In response to the current cache area's usage capacity reaching a preset threshold and the next cache area pointed to by the current cache area being unoccupied, the log data of the current cache area is written to the storage medium and the newly generated log data is set to be written to the next cache area pointed to by the current cache area; The switching of the buffer is implemented through the buffer selector in the data structure. If bufferSelector is LOG_BUFFER_1, the first buffer is used; if it is LOG_BUFFER_2, the second buffer is used; other values ​​are invalid. A log information table is established in the storage medium. The log information table includes the time point when the stored log data was created, the level of the log data, and the module requirements to which the log data belongs. The log information table is stored in a specified area of ​​Flash, with a fixed size of 8 bytes. It is a description of other Flash areas. The log information table contains the time of the earliest log in this page, the log initiating module, and the number of logs of different levels. The time of the earliest log is recorded with a timestamp, and the log initiating module is recorded with a bitmap. Each bit represents a module, and a bit position of 1 indicates that the page contains log data of a certain module. In response to system initialization, the log information table is read from memory; In response to receiving a log read request, the log information table is filtered in memory, and the corresponding log data is read from the storage medium according to the filtering results.

2. The method according to claim 1, characterized in that, Also includes: If the usage capacity of the current cache does not reach a preset threshold within a preset time period, newly generated log data will continue to be written to the current cache.

3. The method according to claim 1, characterized in that, Also includes: In response to the fact that the usage capacity of the current cache area has not reached the preset threshold after a preset time period, it is determined whether the next cache area pointed to by the current cache area is occupied; In response to the fact that the next cache area pointed to by the current cache area is not occupied, the log data of the current cache area is written to the storage medium and the newly generated log data is set to be written to the next cache area pointed to by the current cache area; In response to the next cache area pointed to by the current cache area being occupied, the current cache area is set to wait for the next cache area pointed to by the current cache area to be occupied before being released, and then the log data of the current cache area is written to the storage medium and the newly generated log data is set to be written to the next cache area pointed to by the current cache area.

4. The method according to claim 1, characterized in that, Also includes: In response to the current cache area reaching a preset threshold and the next cache area pointed to by the current cache area being occupied, the current cache area is set to wait for the next cache area pointed to by the current cache area to be released from its occupied state before writing the log data of the current cache area to the storage medium and setting the newly generated log data to be written to the next cache area pointed to by the current cache area.

5. The method according to claim 1, characterized in that, Also includes: Configure flags used to mark the state of the cache; Determining whether the next cache area pointed to by the current cache area is occupied includes: in response to the flag of the next cache area pointed to by the current cache area being not busy, determining that the next cache area pointed to by the current cache area is not occupied; In response to the flag of the next cache area pointed to by the current cache area being busy, it is determined that the next cache area pointed to by the current cache area is occupied.

6. The method according to claim 1, characterized in that, The step of responding to a received log read request by filtering the log information table in memory and reading the corresponding log data from the storage medium according to the filtering results includes: The log data in the log information table is filtered in memory based on the creation time, level, and module requirements of the log data.

7. The method according to claim 1, characterized in that, The settings for multiple caches used for log data caching include: Multiple caches are set up based on the capacity of the storage medium and the latency of each cache.

8. An apparatus for improving the efficiency of log data reading and writing, characterized in that, The device includes: The first module, configuration, is used to set up multiple caches for caching log data; The second module is configured to write log data to the current cache area, determine whether the usage capacity of the current cache area has reached a preset threshold, and determine whether the next cache area pointed to by the current cache area is occupied. The third module is configured to write the log data of the current cache to the storage medium and set the newly generated log data to be written to the next cache pointed to by the current cache in response to the current cache's usage capacity reaching a preset threshold and the next cache pointed to by the current cache being unoccupied. And a module for implementing the following functions: After the log monitoring process starts, it is responsible for monitoring whether the data in the cache needs to be written to Flash: whether a notification to write to Flash is received or whether a timeout has occurred. If no notification to write logs to Flash is received within the specified time, the log monitoring process actively switches the cache and writes the log data in the cache to Flash. The cache switching is implemented through a cache selector in the data structure. If bufferSelector is LOG_BUFFER_1, the first cache area is used; if it is LOG_BUFFER_2, the second cache area is used; other values ​​are invalid values. A log information table is established in the storage medium. The log information table includes the time point when the stored log data was created, the level of the log data, and the module requirements to which the log data belongs. The log information table is stored in a specified area of ​​Flash, with a fixed size of 8 bytes. It is a description of other Flash areas. The log information table contains the time of the earliest log in this page, the log initiating module, and the number of logs of different levels. The time of the earliest log is recorded with a timestamp, and the log initiating module is recorded with a bitmap. Each bit represents a module, and a bit position of 1 indicates that the page contains log data of a certain module. In response to system initialization, the log information table is read from memory; In response to receiving a log read request, the log information table is filtered in memory, and the corresponding log data is read from the storage medium according to the filtering results.

9. A computer device, characterized in that, include: At least one processor; as well as A memory storing computer instructions executable on the processor, which, when executed by the processor, implement the steps of the method according to any one of claims 1-7.

Images