A log storage method, apparatus, device and medium

Abstract

The application provides a log storage method, device, equipment and medium, applied to the technical field of log storage, and comprises the following steps: a log management instance records a log of a data processing task in a target processor core where the log management instance is located, to obtain a first log; the target processor core is a processor core for processing input and output data in a disk array card; a plurality of target processor cores respectively run a log management instance; a plurality of target processor cores respectively run a data processing task; a target data processing path corresponding to the log management instance is determined; the target data processing path is a data processing path bound by the target processor core where the log management instance is located; the first log is transmitted to a target disk mounted by the disk array card for storage based on the target data processing path; and the target disk is a disk in a preset redundant array of independent disks. In this way, the log can be quickly stored, the storage cost is reduced, and the log storage security is improved.

Description

Technical Field

[0001] This invention relates to the field of log storage technology, and in particular to a log storage method, apparatus, device, and medium. Background Technology

[0002] In the field of RAID card systems, software runtime log storage is a crucial element in ensuring system stability, maintainability, and data security. Currently, existing solutions for software runtime log storage in RAID card systems mainly include: directly storing the logs in system memory, which is the fastest method but has a very limited capacity; and using dedicated storage devices, which has the disadvantages of higher cost, requiring additional hardware investment, and the potential for log data loss if the storage device fails.

[0003] Therefore, how to reduce storage costs and improve log storage security while storing logs quickly is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a log storage method, apparatus, device, and medium that can reduce storage costs and improve log storage security while enabling fast log storage. The specific solution is as follows:

[0005] Firstly, this application provides a log storage method, including:

[0006] The log management instance records the logs of the data processing tasks in the target processor core where the log management instance resides, thus obtaining the first log; wherein, the target processor core is the processor core in the disk array card that processes input and output data, the disk array card includes multiple target processor cores, and different target processor cores run log management instances respectively; the data processing task is a task that processes input and output data, and the multiple target processor cores run the data processing task respectively;

[0007] Determine the target data processing path corresponding to the log management instance; wherein, the multiple target processor cores are each bound to different data processing paths, and the target data processing path is the data processing path bound to the target processor core where the log management instance itself resides;

[0008] Based on the target data processing path, the first log is transmitted to the target disk mounted on the disk array card for storage; wherein, the target disk is a disk in a preset independent disk redundancy array.

[0009] Optionally, the first log is transmitted to the target disk mounted on the disk array card for storage based on the target data processing path, including:

[0010] Based on the target data processing path, the first log is transmitted to the target area of ​​the target disk mounted on the disk array card for storage.

[0011] The target disk contains a pre-divided log reservation area, which is further divided into multiple sub-regions. Each sub-region corresponds to a different target processor core, and the target region is the sub-region corresponding to the target processor core of the target data processing path.

[0012] Optionally, the size of each sub-region is determined based on the number of logs generated by the target processor core corresponding to each sub-region.

[0013] Optionally, the disk array card may also have a spare disk in addition to the preset independent redundant disk array. The spare disk includes a pre-defined log reserved area as a spare area. The spare disk is used to replace the faulty disk when any disk in the preset independent redundant disk array fails, and to reconstruct the independent redundant disk array with the disks that have not failed.

[0014] Optionally, logs of data processing tasks within the target processor kernel where the log management instance resides are recorded to obtain the first log, which includes:

[0015] Record the logs of data processing tasks in the target processor kernel where the log management instance resides to a preset log buffer to obtain the first log;

[0016] Accordingly, the first log will be transmitted to the target disk mounted on the disk array card for storage based on the target data processing path, including:

[0017] The first log is read from the preset log buffer and transmitted to the target disk mounted on the disk array card for storage based on the target data processing path.

[0018] Optional, also includes:

[0019] In response to a log read request, the target log corresponding to the log read request is read from the target disk through the target data processing path corresponding to the log management instance.

[0020] Optional, also includes:

[0021] The second log is obtained by receiving log data from other processor cores besides the target processor core through the log management instance;

[0022] Based on the target data processing path corresponding to the log management instance, the two logs are transmitted to the target disk mounted on the disk array card for storage.

[0023] Secondly, this application provides a log storage device, comprising:

[0024] The logging module is used to record the logs of data processing tasks in the target processor core where the log management instance resides through the log management instance, thereby obtaining the first log; wherein, the target processor core is the processor core in the disk array card that processes input and output data, the disk array card includes multiple target processor cores, and different target processor cores run log management instances respectively; the data processing task is a task that processes input and output data, and the multiple target processor cores run the data processing task respectively;

[0025] The path determination module is used to determine the target data processing path corresponding to the log management instance; wherein, the multiple target processor cores are respectively bound to different data processing paths, and the target data processing path is the data processing path bound to the target processor core where the log management instance itself is located;

[0026] The log writing module is used to transmit the first log to the target disk mounted on the disk array card for storage based on the target data processing path; wherein, the target disk is a disk in a preset independent disk redundancy array.

[0027] Thirdly, this application provides an electronic device, comprising:

[0028] Memory, used to store computer programs;

[0029] A processor for executing the computer program to implement the aforementioned log storage steps.

[0030] Fourthly, this application discloses a computer-readable storage medium on which a computer program is stored, and which, when executed by a processor, implements the aforementioned log storage steps.

[0031] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed log storage method.

[0032] As can be seen from the above scheme, this application provides a log storage method, including: recording the logs of data processing tasks in the target processor core where the log management instance itself resides through a log management instance to obtain a first log; wherein, the target processor core is a processor core in a disk array card that processes input and output data, the disk array card includes multiple target processor cores, and different target processor cores respectively run log management instances; the data processing task is a task that processes input and output data, and the multiple target processor cores respectively run the data processing task; determining the target data processing path corresponding to the log management instance; wherein, the multiple target processor cores are respectively bound to different data processing paths, and the target data processing path is the data processing path bound to the target processor core where the log management instance itself resides; and transmitting the first log to a target disk mounted on the disk array card for storage based on the target data processing path; wherein, the target disk is a disk in a preset independent disk redundancy array.

[0033] As can be seen, the beneficial effects of this application are as follows: Log management instances are deployed in the processor cores of each running data processing task. The data processing tasks process input and output data, and the log management instances record the logs of the data processing tasks within their respective target processor cores. The logs are then stored on the target disk mounted on the RAID card using the data processing pathway bound to the target processor core. The target disk is a disk in a pre-defined independent redundant disk array. In other words, this application sets up an independent redundant disk array on the target disk mounted on the RAID card to ensure redundant data storage, avoid data loss, and thus improve data security. Furthermore, multiple log management instances can perform log storage in parallel using their corresponding data processing pathways. The high performance of the input and output data processing pathways in the RAID card ensures fast log storage, and no other dedicated storage devices are required; the logs can be stored on the disk mounted on the RAID card. This reduces storage costs and improves log storage security while enabling fast log storage. Attached Figure Description

[0034] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart of a log storage method provided in an embodiment of this application;

[0036] Figure 2This application provides a schematic diagram of log storage for a disk array card system.

[0037] Figure 3 This is a schematic diagram of a log storage device structure provided in an embodiment of this application;

[0038] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0040] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0041] In the field of RAID card systems, software runtime log storage is a crucial element in ensuring system stability, maintainability, and data security. Its core purpose is to provide system administrators with real-time system monitoring and post-event analysis by recording detailed information during software operation, including operation logs, system status, and anomalies, thereby supporting rapid troubleshooting, performance optimization, and security auditing. However, achieving efficient access and redundancy protection for software runtime log storage in RAID card systems faces numerous technical challenges.

[0042] In the field of RAID card systems, existing solutions for software runtime log storage technology mainly include the following, which aim to ensure efficient access to log data, redundancy protection, and reliable system operation.

[0043] Storing logs directly in system memory is the fastest method, but it limits the amount of logs that can be stored. Using dedicated storage devices (such as SSDs or enterprise-grade hard drives) to store software runtime logs has the disadvantages of higher cost and additional hardware investment. Furthermore, if the storage device fails, log data may be lost.

[0044] To address this, this application provides a log storage solution that can reduce storage costs and improve log storage security while enabling fast log storage.

[0045] First, the terms used in this application will be explained:

[0046] RAID (Redundant Array of Independent Disks), often simply referred to as a disk array, is a technology that uses multiple independent, high-performance disk drives as a subsystem to provide higher storage performance and data redundancy than a single disk.

[0047] RAID 1: A RAID level. It writes data identically to both the working disk and the mirror disk, utilizing up to 50% of disk space. RAID 1 provides optimal data protection; if the working disk fails, the system automatically reads data from the mirror disk, ensuring uninterrupted user operation.

[0048] RAID card: Short for Redundant Independent Disk Array card, it's a technology that combines multiple hard drives into a single logical drive, designed to improve storage capacity, performance, reliability, and manageability. It combines hard drives through different RAID levels, providing higher storage performance and data redundancy than a single hard drive.

[0049] Parallelism: Parallelism refers to two or more events or activities occurring at the same time, that is, multiple tasks are executed simultaneously on multiple processors or processing units, without competition or waiting for CPU resources.

[0050] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] Next, a log storage method provided by an embodiment of the present invention will be described in detail. Figure 1 This invention provides a flowchart of a log storage method, which includes:

[0052] Step S11: Record the logs of the data processing tasks in the target processor core where the log management instance resides through the log management instance to obtain the first log; wherein, the target processor core is the processor core in the disk array card that processes input and output data, the disk array card includes multiple target processor cores, and different target processor cores run log management instances respectively; the data processing task is a task that processes input and output data, and the multiple target processor cores run the data processing tasks respectively.

[0053] In this embodiment, multiple log management instances can be pre-created and run on various target processors. In one optional implementation, the log management instance can run as an independent component; in another optional implementation, it can be a component within a data processing task. Input / output data refers to I / O data. The target processor core is the processor core in the disk array card that processes I / O data. The disk array card also includes other processor cores that are not used for processing input / output data, such as cores used for business management and configuration.

[0054] Step S12: Determine the target data processing path corresponding to the log management instance; wherein, the multiple target processor cores are respectively bound to different data processing paths, and the target data processing path is the data processing path bound to the target processor core where the log management instance itself is located.

[0055] In one optional implementation, the target data processing path corresponding to the log management instance can be determined based on a preset mapping relationship. Embodiments of this application can configure a preset mapping relationship for the log management instance, in which the log management instance can correspond one-to-one with a data processing path. A data processing path is a component that processes and transmits data.

[0056] Step S13: Based on the target data processing path, the first log is transmitted to the target disk mounted on the disk array card for storage; wherein, the target disk is a disk in a preset independent disk redundancy array.

[0057] This application embodiment can create an independent redundant disk array based on the disks attached to the disk array card, resulting in a preset independent redundant disk array, such as RAID1 or RAID2. Redundant storage of logs can be achieved through this preset independent redundant disk array.

[0058] In one optional implementation, the first log can be transmitted to a target area on the target disk mounted on the disk array card for storage based on the target data processing path; wherein, the target disk has a pre-divided log reservation area, and the log reservation area is pre-divided into multiple sub-regions, each sub-region corresponding to a different target processor core, and the target area is the sub-region corresponding to the target processor core corresponding to the target data processing path.

[0059] That is, in the embodiments of this application, each disk in the preset independent disk redundant array is divided into a log reservation area, which is dedicated to storing logs. The logs corresponding to different target processor cores can be stored in different sub-regions of the log reservation area, thereby avoiding competition caused by multi-way storage, realizing parallel storage, and improving storage performance.

[0060] Furthermore, in an optional implementation, the size of each sub-region is determined based on the number of logs generated by the target processor core corresponding to each sub-region. The number of logs is obtained by counting the number of log records within a preset time period. This allows for more efficient use of storage space and improves storage performance.

[0061] Furthermore, the disk array card also has a spare disk in addition to the preset independent disk redundancy array. The spare disk includes a pre-divided log reserved area as a spare area. The spare disk is used to replace the faulty disk when any disk in the preset independent disk redundancy array fails, and to reconstruct the independent disk redundancy array with the disks that have not failed.

[0062] The spare area is also pre-divided into multiple sub-regions, each corresponding to a different target processor core. The target region is the sub-region corresponding to the target processor core of the target data processing path, in order to ensure stable replacement of the faulty disk.

[0063] In an alternative implementation, the sub-regions may also be evenly divided into log reservation areas.

[0064] In other words, in addition to using a redundant array of independent disks to achieve redundant log storage and ensure data security, this application embodiment can also use a spare disk to replace a faulty disk in a timely manner to ensure log storage security.

[0065] In one optional implementation, recording the logs of data processing tasks in the target processor kernel where the log management instance resides to obtain a first log may include: recording the logs of data processing tasks in the target processor kernel where the log management instance resides to a preset log buffer to obtain a first log; correspondingly, transmitting the first log to the target disk mounted on the disk array card for storage based on the target data processing path includes: reading the first log from the preset log buffer and transmitting the first log to the target disk mounted on the disk array card for storage based on the target data processing path.

[0066] That is, embodiments of this application can also set up a log buffer as a temporary log storage area. The buffer can be a region in memory used to store and transmit data, so as to provide convenience during data transmission and processing. The preset log buffer is used to temporarily store logs, which can smooth data transmission and make data coordinated between different processing speeds.

[0067] In an optional implementation, in response to a log read request, the target log corresponding to the log read request can be read from the target disk via the target data processing path corresponding to the log management instance. This embodiment of the application can provide a log read interface in the log management instance and reuse the data processing path to improve log read efficiency.

[0068] In an optional implementation, log data from other processor cores besides the target processor core can be received through a log management instance to obtain a second log; the second log is then transmitted to the target disk mounted on the disk array card for storage based on the target data processing path corresponding to the log management instance.

[0069] In other words, the log data of other processors' kernels can be stored using the log management instance and data processing path described in this application. The kernels of these other processors can be non-target processor kernels, i.e., processor kernels that do not run data processing tasks, such as business management and configuration kernels.

[0070] In an optional implementation, if there are N non-target processor cores and M target processor cores, where N can be one or more, if N is less than or equal to M, then each non-target processor core can be bound to a target processor core. If N is greater than M, then based on a uniform distribution principle, each non-target processor core is bound to a target processor core. The uniform distribution principle is that the difference between the number of non-target processor cores bound to each target processor core is less than or equal to a preset value, such as 1, meaning the numbers are equal or differ by 1. In this way, the logs of the non-target processor cores can be quickly stored using the target processor cores.

[0071] In an optional implementation, the target processor cores can be bound separately to the non-target processor cores based on the number of logs for each target processor core. The number of logs for each target processor core can be sorted, and the non-target processor cores can be bound to the target processor cores with smaller number of logs first, so that the total amount of logs processed by each log management instance is more even and the log storage is more stable.

[0072] In one optional implementation, this application embodiment can utilize a target data processing path corresponding to a determined log management instance; based on the target data processing path, the first log is transmitted to a target disk mounted on the disk array card for storage; wherein, the target disk is a disk in a preset independent disk redundancy array.

[0073] As can be seen, in this embodiment, log management instances are deployed in the processor cores of each running data processing task. The data processing tasks process input and output data, and the log management instances record the logs of the data processing tasks within their respective target processor cores. The logs are then stored on the target disk mounted on the RAID card using the data processing pathway bound to the target processor core. The target disk is a disk in a pre-defined independent redundant disk array. In other words, this application sets up an independent redundant disk array on the target disk mounted on the RAID card to ensure redundant data storage, prevent data loss, and improve data security. Furthermore, multiple log management instances can perform log storage in parallel using their corresponding data processing pathways. The high performance of the input / output data processing pathways in the RAID card ensures fast log storage without the need for other dedicated storage devices; the logs can be stored on the disk mounted on the RAID card. This reduces storage costs and improves log storage security while enabling fast log storage.

[0074] Further, see Figure 2 As shown in the figure, this application embodiment provides a log storage diagram of a disk array card system. The disk array card system, or RAID card system, consists of two parts: the RAID card (i.e., disk array card 0 and the disks connected to the RAID card). The RAID card part is further divided into a software part and a hardware part. The software part is executed by a multi-core CPU. These CPU cores (i.e., processor cores) are divided into two parts: CPU cores for processing I / O data and cores for business management and configuration. The CPU cores are hardware, and the software part in the figure represents the tasks running on the corresponding CPU cores. The number of CPU cores for processing I / O data is equal to the number of I / O processing channels, and a task is created for each core, which exclusively occupies the corresponding CPU core. Based on each data processing core, a task is created... A log instance is created to access the logs of tasks running on its corresponding core, control the logging behavior, and manage the corresponding buffer. The hardware has four I / O processing channels, all with identical functions. Each I / O channel is used for RAID calculations of I / O data and disk access operations, and the processing of log data is also the same. On the disk side, a log reservation area is allocated to each disk connected to the RAID card for storing and retrieving log data. Within the log reservation area, four regions are further divided, each corresponding to a core that processes I / O data, used to record the logs generated during I / O data processing. Since the logs reside on the disk side, to address the issue of log loss due to disk failures or drops, this application protects the log reservation area using RAID1, such as... Figure 2In this configuration, the reserved area of ​​disks not included in RAID 1 is used as a backup area for backing up data in the event of disk failure or damage. This embodiment uses four CPU cores as an example, but is not limited to four CPU cores. Logs from other processor cores can be sent in the form of msg messages.

[0075] This application embodiment divides the CPU cores in the system into two groups: one group for business management and configuration, and the other for I / O data processing. Taking a RAID system with four cores as an example, there are four RAID acceleration computing clusters, which can be considered as four hardware I / O processing paths. To greatly improve the overall parallel I / O processing capability of the system and minimize mutual exclusion access to shared hardware and software resources, each of the four CPU cores is bound to one of the four hardware I / O processing paths. On each of these four cores, a task is started to process user I / O data and I / O data (log data) generated by the RAID card itself. The other CPU cores are used for business management and configuration. For the group of CPU cores processing I / O data, a log management instance is created based on the CPU core. The log management instance provides interfaces for recording logs, reading logs, and querying log configurations for other modules that need to record logs. Internally, based on the core where the current log management instance resides, the corresponding hardware I / O processing path is configured to complete the RAID calculation and disk access of log data.

[0076] Allocate reserved space for log storage on each disk of the RAID card. Select two normal disks and combine their reserved areas into a RAID 1 array. The reserved areas on the other disks are used as hot spare reserved areas. RAID 1 offers the highest security level and is the simplest to implement in a storage system, making it ideal for storing critical data like logs. Furthermore, due to the reuse of hardware RAID for accelerated computation and disk write functionality, RAID 1 offers high performance in log access, truly achieving secure and efficient log storage. The drawback of RAID 1 is that it only utilizes about 50% of the disk capacity. However, for disks with several terabytes of storage, allocating several gigabytes of reserved space for logs is relatively easy. And several gigabytes is more than enough for log recording. The reserved areas are divided according to the number of CPU cores processing I / O data; the size of the reserved area allocated to each core is estimated based on the amount of logs generated by that core.

[0077] Since four I / O processing paths share a single RAID array, in terms of hardware implementation, there's a possibility that these four paths will contend for the same I / O space resource within the RAID array. This contention degrades I / O read and write performance. To avoid this contention, the log storage space is divided into four parts, ensuring true parallelism of log recording across all four cores. If one disk in the RAID 1 array used for log storage fails, it can be replaced using the reserved space from other disks.

[0078] Although RAID1 offers a high level of data security, it is still susceptible to disk failure. To achieve a high level of redundancy protection, this embodiment of the application sets up multiple backup disks to replace disks that may fail in RAID1.

[0079] In terms of efficient access, the high concurrency and large data volumes of RAID card systems place extremely high demands on the performance of log storage systems. Efficient access not only means that log data can be written and retrieved in real time and quickly, but also requires stable performance under high load. To achieve this goal, efficient log writing mechanisms, such as asynchronous writing and batch writing, are needed to reduce the number of disk I / O operations and latency. Simultaneously, a reasonable storage structure and indexing strategy need to be designed to accelerate log data retrieval. Regarding redundancy protection, while RAID card systems themselves possess data redundancy and fault tolerance capabilities, redundancy protection for software runtime logs is equally important. Since log data plays a crucial role in system recovery and troubleshooting, its loss or damage will severely impact system maintenance. Therefore, redundant storage of log data is necessary to ensure that log data can be quickly recovered from backup media in the event of a primary storage media failure.

[0080] Therefore, this application provides a fast and reliable log storage solution for RAID card systems. It leverages the multi-core CPU and hardware-provided parallel capabilities of the RAID card system to improve end-to-end log access performance, and utilizes RAID's data redundancy and fault tolerance to enhance the security of stored log data. This solution fully utilizes RAID's redundancy protection capabilities to safeguard log data. Furthermore, it achieves end-to-end parallel log storage, significantly improving log access performance and enhancing the debugging and maintenance capabilities of the RAID card system. This application fully utilizes the RAID card's own hardware advantages in efficient data access, treating the log data generated by the RAID card software as data generated by the RAID card itself, and designs a fast and reliable log data access solution to maximize the debugging and maintenance capabilities of the RAID card system.

[0081] This invention proposes a fast and reliable log storage solution for RAID card systems. Of course, this method can also be used in non-storage systems implemented using the solution described in this application.

[0082] Further, see Figure 3 As shown, Figure 3 This is a schematic diagram of a log storage device provided in an embodiment of the present invention. The log storage device includes:

[0083] The log recording module 31 is used to record the logs of the data processing tasks in the target processor core where the log management instance is located through the log management instance, and obtain the first log; wherein, the target processor core is the processor core in the disk array card that processes input and output data, the disk array card includes multiple target processor cores, and different target processor cores run log management instances respectively; the data processing task is a task that processes input and output data, and the multiple target processor cores run the data processing task respectively;

[0084] The path determination module 32 is used to determine the target data processing path corresponding to the log management instance; wherein, the multiple target processor cores are respectively bound to different data processing paths, and the target data processing path is the data processing path bound to the target processor core where the log management instance itself is located;

[0085] The log writing module 33 is used to transmit the first log to the target disk mounted on the disk array card for storage based on the target data processing path; wherein, the target disk is a disk in a preset independent disk redundancy array.

[0086] In an optional implementation, the log writing module 33 is specifically used for:

[0087] Based on the target data processing path, the first log is transmitted to the target area of ​​the target disk mounted on the disk array card for storage.

[0088] The target disk contains a pre-divided log reservation area, which is further divided into multiple sub-regions. Each sub-region corresponds to a different target processor core, and the target region is the sub-region corresponding to the target processor core of the target data processing path.

[0089] In an optional implementation, the size of each sub-region is determined based on the number of logs generated by the target processor core corresponding to each sub-region.

[0090] In an optional implementation, the disk array card also has a spare disk in addition to the preset independent disk redundancy array. The spare disk includes a pre-divided log reserved area as a spare area. The spare disk is used to replace the faulty disk when any disk in the preset independent disk redundancy array fails, and to reconstruct the independent disk redundancy array with the disks that have not failed.

[0091] In an optional implementation, the log recording module 31 can be specifically used to record the logs of the data processing tasks in the target processor kernel where the log management instance is located to a preset log buffer to obtain the first log; correspondingly, the log disk writing module 33 is specifically used to: read the first log from the preset log buffer, and transmit the first log to the target disk mounted on the disk array card for storage based on the target data processing path.

[0092] Furthermore, in an optional embodiment, the device further includes:

[0093] The log read response module is used to respond to a log read request by reading the target log corresponding to the log read request from the target disk through the target data processing path corresponding to the log management instance.

[0094] In an optional implementation, the log recording module 31 is further configured to receive log data from other processor cores besides the target processor core through the log management instance to obtain a second log; correspondingly, the log disk writing module 33 can also be configured to transmit the second log to the target disk mounted on the disk array card for storage based on the target data processing path corresponding to the log management instance.

[0095] As can be seen, in this embodiment, log management instances are deployed in the processor cores of each running data processing task. The data processing tasks process input and output data, and the log management instances record the logs of the data processing tasks within their respective target processor cores. The logs are then stored on the target disk mounted on the RAID card using the data processing pathway bound to the target processor core. The target disk is a disk in a pre-defined independent redundant disk array. In other words, this application sets up an independent redundant disk array on the target disk mounted on the RAID card to ensure redundant data storage, prevent data loss, and improve data security. Furthermore, multiple log management instances can perform log storage in parallel using their corresponding data processing pathways. The high performance of the input / output data processing pathways in the RAID card ensures fast log storage without the need for other dedicated storage devices; the logs can be stored on the disk mounted on the RAID card. This reduces storage costs and improves log storage security while enabling fast log storage.

[0096] Figure 3 For a description of the features in the corresponding embodiments, please refer to Figure 1 The relevant descriptions of the corresponding embodiments will not be repeated here.

[0097] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 4As shown, the electronic device includes: a memory 40 for storing computer programs;

[0098] The processor 41 is used to implement the steps of the log storage method as described in the above embodiments when executing a computer program.

[0099] The processor 41 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 41 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 41 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 41 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 41 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0100] The memory 40 may include one or more computer-readable storage media, which may be non-transitory. The memory 40 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 40 is used to store at least the following computer program 401, which, after being loaded and executed by the processor 41, is capable of implementing the relevant steps of the log storage method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, and the storage method may be temporary or permanent storage. The operating system 402 may include Windows, Unix, Linux, etc. The data 403 may include, but is not limited to, log data.

[0101] In some embodiments, the electronic device may further include a display screen 42, an input / output interface 43, a communication interface 44, a power supply 45, and a communication bus 46.

[0102] Those skilled in the art will understand that Figure 4The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0103] It is understood that if the log storage method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk, or optical disk, and other media capable of storing program code.

[0104] Based on this, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the log storage method described above.

[0105] The following describes a computer program product provided by an embodiment of this application. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0106] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed log storage method.

[0107] The foregoing has provided a detailed description of a log storage method, apparatus, device, and medium provided by embodiments of the present invention. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0108] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0109] The above provides a detailed description of the log storage method, apparatus, device, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A log storage method, characterized in that, include: The log management instance records the logs of the data processing tasks in the target processor core where the log management instance resides, thus obtaining the first log; wherein, the target processor core is the processor core in the disk array card that processes input and output data, the disk array card includes multiple target processor cores, and different target processor cores run log management instances respectively; the data processing task is a task that processes input and output data, and the multiple target processor cores run the data processing task respectively; Determine the target data processing path corresponding to the log management instance; wherein, the multiple target processor cores are each bound to different data processing paths, and the target data processing path is the data processing path bound to the target processor core where the log management instance itself resides; Based on the target data processing path, the first log is transmitted to the target disk mounted on the disk array card for storage; wherein, the target disk is a disk in a preset independent disk redundancy array; The method further includes: Log data from other processor cores besides the target processor core is received through the log management instance to obtain the second log; the second log is transmitted to the target disk mounted on the disk array card for storage based on the target data processing path corresponding to the log management instance; the number of logs for each target processor core is sorted, and other processor cores are bound to the target processor core with the smaller number of logs.

2. The log storage method according to claim 1, characterized in that, Based on the target data processing path, the first log is transmitted to the target disk mounted on the disk array card for storage, including: Based on the target data processing path, the first log is transmitted to the target area of ​​the target disk mounted on the disk array card for storage. The target disk contains a pre-divided log reservation area, which is further divided into multiple sub-regions. Each sub-region corresponds to a different target processor core, and the target region is the sub-region corresponding to the target processor core of the target data processing path.

3. The log storage method according to claim 2, characterized in that, The size of each subregion is determined based on the number of logs generated by the target processor kernel corresponding to each subregion.

4. The log storage method according to claim 2, characterized in that, The disk array card also has spare disks in addition to the preset independent disk redundancy array. The spare disks include a pre-divided log reserved area as a spare area. The spare disks are used to replace the faulty disk when any disk in the preset independent disk redundancy array fails, and to reconstruct the independent disk redundancy array with the disks that have not failed.

5. The log storage method according to claim 1, characterized in that, The logs of data processing tasks within the target processor kernel where the log management instance resides are recorded, resulting in the first log, which includes: Record the logs of data processing tasks in the target processor kernel where the log management instance resides to a preset log buffer to obtain the first log; Accordingly, the first log will be transmitted to the target disk mounted on the disk array card for storage based on the target data processing path, including: The first log is read from the preset log buffer and transmitted to the target disk mounted on the disk array card for storage based on the target data processing path.

6. The log storage method according to claim 1, characterized in that, Also includes: In response to a log read request, the target log corresponding to the log read request is read from the target disk through the target data processing path corresponding to the log management instance.

7. A log storage device, characterized in that, include: The logging module is used to record the logs of data processing tasks in the target processor core where the log management instance resides through the log management instance, thereby obtaining the first log; wherein, the target processor core is the processor core in the disk array card that processes input and output data, the disk array card includes multiple target processor cores, and different target processor cores run log management instances respectively; the data processing task is a task that processes input and output data, and the multiple target processor cores run the data processing task respectively; The path determination module is used to determine the target data processing path corresponding to the log management instance; wherein, the multiple target processor cores are respectively bound to different data processing paths, and the target data processing path is the data processing path bound to the target processor core where the log management instance itself is located; The log writing module is used to transmit the first log to the target disk mounted on the disk array card for storage based on the target data processing path; wherein, the target disk is a disk in a preset independent disk redundancy array; The device is also used for: Log data from other processor cores besides the target processor core is received through the log management instance to obtain the second log; the second log is transmitted to the target disk mounted on the disk array card for storage based on the target data processing path corresponding to the log management instance; the number of logs for each target processor core is sorted, and other processor cores are bound to the target processor core with the smaller number of logs.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the log storage step as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the log storage steps as described in any one of claims 1 to 6.

Images