Log processing method and device, electronic equipment, storage medium and program product

Abstract

The application provides a log processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: receiving a processing request for target data, the processing request comprising position information of the target data; determining a plurality of log units in a storage device, the log units being used for independent task processing; performing hash operation processing on the position information to determine identification information corresponding to the processing request, the identification information being used for indicating a log unit to which the processing request is allocated; and determining a target log unit from the plurality of log units according to the identification information, and writing the processing request into a storage space corresponding to the target log unit. In this way, the position information can be extracted according to the processing request, the target log unit corresponding to the processing request can be determined through hash operation processing, and accurate allocation can be performed. Meanwhile, the performance bottleneck problem of the traditional single log stream mode can be solved through the cooperation of the hash operation processing algorithm and the plurality of independent log units.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to a log processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the explosive growth of data volume and the stringent business requirements for high throughput and low latency, the single log stream processing mode can no longer meet performance requirements.

[0003] In related technologies, all transactions are written serially through a single log stream, thus ensuring transaction order and consistency. However, in the above methods, log writing relies on fixed thread allocation, which cannot utilize the concurrency capabilities of multi-core processors and multi-disk drives, thereby limiting the throughput of the storage system and leading to performance bottlenecks. Summary of the Invention

[0004] This application provides a log processing method, apparatus, electronic device, storage medium, and program product to solve the performance bottleneck problem of storage systems in single log stream mode.

[0005] Firstly, this application provides a log processing method, including:

[0006] Receive a processing request for target data, the processing request including the location information of the target data;

[0007] Multiple log units are defined within the storage device, and each log unit is used for independent task processing.

[0008] The location information is hashed to determine the identification information corresponding to the processing request. The identification information is used to indicate the log unit allocated to the processing request.

[0009] Based on the identification information, the target log unit is determined among multiple log units, and the processing request is written to the storage space corresponding to the target log unit.

[0010] In this embodiment, location information can be extracted based on the processing request, and the target log unit corresponding to the processing request can be determined through hash operation, thereby enabling accurate allocation. At the same time, the performance bottleneck problem of the traditional single log stream mode is solved by the collaboration of hash operation processing algorithm and multiple independent log units.

[0011] Optionally, the method described above performs a hash operation on the location information to determine the identification information corresponding to the processing request, including:

[0012] Based on the location information, determine the storage volume number corresponding to the target data;

[0013] Determine the number of units for multiple log units;

[0014] Based on the number of units and the storage volume number, determine the identification information corresponding to the processing request.

[0015] In this embodiment, the storage volume number can be determined through location information. Hash operations are performed based on the storage volume number and the number of units to determine the identification information. Through hash operations, the outputs of different inputs can be evenly distributed across multiple log units.

[0016] Optionally, the method described above determines the identification information corresponding to the processing request based on the number of units and the storage volume number, including:

[0017] Based on the number of units, the storage volume number is processed by taking a modulo to obtain the first result of the modulo processing;

[0018] The first result is determined as the identification information corresponding to the processing request.

[0019] In this embodiment, in a multi-volume scenario, requests from different volumes can be allocated to different log units, thereby achieving volume-level resource isolation. Furthermore, all requests from the same storage volume are allocated to the same unit, which facilitates subsequent data reconstruction, flushing, and other operations, thereby ensuring data consistency.

[0020] Optionally, the method described above, which determines the identification information corresponding to the processing request based on the number of units and the storage volume number, further includes:

[0021] When the number of storage volumes is 1, obtain the preset data granularity. The data granularity is used to divide the storage space within the storage volume.

[0022] Based on the location information, the logical block address corresponding to the target data is determined. The logical block address is used to indicate the offset of the target data within the storage volume.

[0023] The identification information corresponding to the processing request is determined based on the data granularity, logical block address, and storage volume number.

[0024] In this embodiment, in a single-volume scenario, requests for different data blocks within a single volume can be allocated to different log units, thereby enabling parallel processing within a single volume, making full use of the bandwidth of multiple log hard disks, and the data granularity can be adjusted according to the scenario to adapt to different business scenarios.

[0025] Optionally, the method described above, after writing the processing request to the storage space corresponding to the target log unit, further includes:

[0026] Receive rebuild instructions for multiple log units; the rebuild instructions are generated after the storage device restarts.

[0027] Retrieve log information corresponding to multiple log units;

[0028] Based on the log information, determine the valid range corresponding to multiple log units;

[0029] Based on the valid intervals corresponding to multiple log units, multiple log units are reconstructed in parallel.

[0030] In this embodiment, parallel reconstruction processing can refer to allocating independent reconstruction task threads to each log unit. The reconstruction process of each log unit is carried out simultaneously and without interference, which can improve reconstruction efficiency compared to serial reconstruction.

[0031] Optionally, using the method described above, based on the log information, for any given log unit, determine the valid range corresponding to multiple log units, including:

[0032] Retrieve log information from log units;

[0033] Within the log information, determine the first offset and the second offset. The first offset is used to indicate the current data writing position of the log unit, and the second offset is used to indicate the position where the log unit reclaims the data.

[0034] Based on the first offset and the second offset, the position interval between the first offset and the second offset is determined as the valid interval corresponding to the log unit.

[0035] In this embodiment, the effective range is defined by the offset, and only the core data within the effective range that needs to be recovered is processed, avoiding the reconstruction of irrelevant data, thereby improving the reconstruction efficiency.

[0036] Optionally, the method described above, after writing the processing request to the storage space corresponding to the target log unit, further includes:

[0037] Obtain status information for multiple log units; the status information is used to indicate the resource occupancy status of the log units.

[0038] Adjustments are made to multiple log units based on status information.

[0039] In this embodiment, resource elastic allocation is achieved by dynamically adjusting the number of log instances, which solves the problem of resource idleness or overload caused by the traditional fixed number of instances. This ensures that the system is always in the optimal load range, avoids performance jitter caused by overload of a single log unit, and prevents waste caused by idle resources. This enables the system to maintain performance stability in both high concurrency and low load scenarios, and improves resource utilization and business continuity.

[0040] Optionally, the above method can be used to adjust multiple log units based on status information, including:

[0041] Based on the status information, determine the data request frequency of multiple log units;

[0042] Obtain the resource utilization rate of storage devices;

[0043] Based on the resource utilization of the storage device and the data request frequency of multiple log units, determine the load parameters corresponding to multiple log units. The load parameters are used to indicate the load level of the log units.

[0044] For any log unit, if the load parameter corresponding to the log unit is less than or equal to a preset threshold, the log unit is merged.

[0045] When the load parameter corresponding to a log unit exceeds a preset threshold, the log unit is split.

[0046] In this embodiment, weighted calculation is used to achieve more accurate dynamic adjustment, which solves the problem of misjudgment caused by traditional single indicators. By comprehensively evaluating the load status through multi-dimensional indicators, the accuracy of dynamic adjustment is ensured.

[0047] Secondly, this application provides a log processing apparatus, comprising:

[0048] The receiving module is used to receive processing requests for target data, including the location information of the target data.

[0049] The first determination module is used to determine multiple log units within the storage device, and each log unit is used for independent task processing.

[0050] The second determining module is used to perform hash operation on the location information to determine the identification information corresponding to the processing request. The identification information is used to indicate the log unit allocated to the processing request.

[0051] The write module is used to determine the target log unit among multiple log units based on the identification information, and write the processing request to the storage space corresponding to the target log unit.

[0052] Optionally, in the above apparatus, the second determining module is specifically used for,

[0053] Based on the location information, determine the storage volume number corresponding to the target data;

[0054] Determine the number of units for multiple log units;

[0055] Based on the number of units and the storage volume number, determine the identification information corresponding to the processing request.

[0056] Optionally, in the above apparatus, the second determining module is specifically used for,

[0057] Based on the number of units, the storage volume number is processed by taking a modulo to obtain the first result of the modulo processing;

[0058] The first result is determined as the identification information corresponding to the processing request.

[0059] Optionally, in the above apparatus, the second determining module is specifically used for,

[0060] When the number of storage volumes is 1, obtain the preset data granularity. The data granularity is used to divide the storage space within the storage volume.

[0061] Based on the location information, the logical block address corresponding to the target data is determined. The logical block address is used to indicate the offset of the target data within the storage volume.

[0062] The identification information corresponding to the processing request is determined based on the data granularity, logical block address, and storage volume number.

[0063] Optionally, the above-described apparatus further includes a reconstruction processing module, which is used for:

[0064] Receive rebuild instructions for multiple log units; the rebuild instructions are generated after the storage device restarts.

[0065] Retrieve log information corresponding to multiple log units;

[0066] Based on the log information, determine the valid range corresponding to multiple log units;

[0067] Based on the valid intervals corresponding to multiple log units, multiple log units are reconstructed in parallel.

[0068] Optionally, in the above-described apparatus, the reconstruction processing module is specifically used for,

[0069] Retrieve log information from log units;

[0070] Within the log information, determine the first offset and the second offset. The first offset is used to indicate the current data writing position of the log unit, and the second offset is used to indicate the position where the log unit reclaims the data.

[0071] Based on the first offset and the second offset, the position interval between the first offset and the second offset is determined as the valid interval corresponding to the log unit.

[0072] Optionally, the above-described device further includes an adjustment module, which is used for,

[0073] Obtain status information for multiple log units; the status information is used to indicate the resource occupancy status of the log units.

[0074] Adjustments are made to multiple log units based on status information.

[0075] Optionally, in the above-described device, the adjustment module is specifically used for,

[0076] Based on the status information, determine the data request frequency of multiple log units;

[0077] Obtain the resource utilization rate of storage devices;

[0078] Based on the resource utilization of the storage device and the data request frequency of multiple log units, determine the load parameters corresponding to multiple log units. The load parameters are used to indicate the load level of the log units.

[0079] For any log unit, if the load parameter corresponding to the log unit is less than or equal to a preset threshold, the log unit is merged.

[0080] When the load parameter corresponding to a log unit exceeds a preset threshold, the log unit is split.

[0081] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0082] The memory stores the instructions that the computer executes;

[0083] The processor executes computer-executable instructions stored in memory to implement the method as described in the first aspect.

[0084] The electronic device provided in this application embodiment can execute the technical solutions in the above method embodiments, and its beneficial effects are similar, so they will not be described again here.

[0085] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a computer, are used to implement the method as described in the first aspect.

[0086] The computer-readable storage medium provided in this application embodiment can execute the technical solutions in the above method embodiments, and its beneficial effects are similar, so they will not be described again here.

[0087] Fifthly, this application provides a computer program product, including a computer program, which, when executed by a computer, is used to implement the method of the first aspect.

[0088] The computer program product provided in this application embodiment can execute the technical solutions in the above method embodiments, and its beneficial effects are similar, so they will not be described again here.

[0089] The log processing method, apparatus, electronic device, storage medium, and program product provided in this application, when log processing is required, can receive processing requests for target data, including the location information of the target data; determine multiple log units within the storage device, each log unit being used for independent task processing; perform hash operations on the location information to determine identification information corresponding to the processing request, the identification information indicating the log unit allocated to the processing request; determine the target log unit among the multiple log units based on the identification information, and write the processing request to the storage space corresponding to the target log unit. Thus, through the above method, location information can be extracted based on the processing request, and the target log unit corresponding to the processing request can be determined through hash operations, thereby enabling precise allocation; simultaneously, the performance bottleneck problem of the traditional single log stream mode is solved through the collaboration of the hash operation algorithm and multiple independent log units. Attached Figure Description

[0090] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0091] Figure 1 This is a schematic diagram of an example log processing architecture in this application;

[0092] Figure 2 A flowchart illustrating a log processing method provided in an embodiment of this application;

[0093] Figure 3 This is a schematic diagram illustrating the process of determining identification information provided in an embodiment of this application;

[0094] Figure 4 A schematic diagram of the log unit reconstruction process provided in the embodiments of this application;

[0095] Figure 5 A schematic diagram illustrating the adjustment process of the log unit provided in an embodiment of this application;

[0096] Figure 6 A flowchart illustrating the complete log processing method provided in the application embodiment;

[0097] Figure 7 This is a schematic diagram of the structure of a log processing device provided in an embodiment of this application;

[0098] Figure 8 This is a schematic diagram of another log processing device provided in an embodiment of this application;

[0099] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0100] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0101] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0102] In high-concurrency, high-throughput distributed storage systems, such as cloud computing platforms, large-scale database clusters, and real-time data processing systems, the storage engine needs to handle massive transactional operations, such as financial transactions and real-time data analysis. The system must maximize throughput while ensuring data consistency and durability. For example, in a financial transaction system, tens of thousands of transaction requests need to be processed per second, and each transaction must be logged to ensure atomicity. In a distributed database, cross-node data synchronization and fault recovery rely on reliable log writing and rapid reconstruction.

[0103] In related technologies, the single-log stream scheme can process transaction logs serially through a single thread, which can guarantee the order of transactions, but it cannot utilize the parallel resources of multi-core CPUs and multiple storage devices, resulting in performance limitations.

[0104] Specifically, log writing relies on fixed thread allocation, which cannot utilize the concurrency capabilities of multi-core CPUs, resulting in limited throughput. When the system is equipped with multiple high-speed storage devices (such as SSDs / NVMs), a single log stream cannot distribute the write load across multiple hard drives, causing hardware resources to be idle. In single-log instance mode, the entire log file needs to be scanned sequentially upon system restart, and the recovery time is proportional to the log size, making it difficult to meet the rapid recovery requirements of large-scale data scenarios. For example, in financial trading scenarios, single-threaded writing cannot meet the processing requirements of tens of thousands of transactions per second, and the logs need to be rebuilt line by line after system restart, which is time-consuming.

[0105] While multi-log stream solutions can improve concurrency, they require maintaining global transaction dependencies, increasing system complexity and overhead.

[0106] Specifically, multi-stream log solutions improve performance through parallel writing, but require additional mechanisms to maintain transaction dependencies, increasing implementation complexity. For example, if transaction A and transaction B are dependent, but transaction A writes to log 1 while transaction B writes to log 2, the commit order of the two needs to be coordinated; otherwise, data consistency may be compromised. Furthermore, a fixed number of log streams is difficult to adapt to different business loads, such as multi-volume concurrency or single-volume high-throughput scenarios, leading to low resource utilization.

[0107] Therefore, this application provides a log processing method that splits a single log instance into multiple logically independent log instances, each instance being responsible for transaction recording of a specific volume device or data segment, and dynamically allocates transactions to different instances through a hash algorithm, thereby achieving parallel writing of multiple threads, concurrent requests from multiple storage devices, and parallelization of log reconstruction tasks while ensuring transaction consistency.

[0108] To facilitate understanding, the following will be combined with... Figure 1 The application scenarios applicable to the embodiments of this application will be described.

[0109] Figure 1 This is a schematic diagram of an example log processing architecture in this application, such as... Figure 1 As shown, the system includes a user device 101 and a storage device 102. The user device 101 can send a processing request for target data to the storage device 102, so that the storage device 102 can process the processing request. Specifically, the user device 101 can send a data read or data write request to the storage device 102, so that the storage device 102 can write data or send data to the user device 101.

[0110] For example, in a financial transaction system, tens of thousands of transaction requests need to be processed per second, and each transaction needs to be logged through storage device 102 to ensure atomicity; in a distributed database, cross-node data synchronization and fault recovery depend on the reliable writing and rapid reconstruction of the logs of storage device 102.

[0111] The storage device 102 may include multiple log units, each of which can be an independent log recording entity used to record logs, i.e., event information of operations. For example, the log may include timestamps, log levels, event subjects, event descriptions, etc.

[0112] The multiple log units within storage device 102 record logs, which facilitates rapid identification of the root cause of failures, quantitative optimization of performance, flexible management and control, and meeting security requirements.

[0113] In one possible implementation, the specific application scenarios of this application may include financial scenarios, cloud storage scenarios, big data scenarios, database cluster scenarios, distributed storage scenarios, etc.

[0114] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0115] Figure 2 This is a flowchart illustrating a log processing method provided in an embodiment of this application. The method can be executed by a log processing device, which can be implemented using a computer program; it can also be implemented using a medium storing the relevant computer program, such as a USB flash drive and / or optical disc; or it can be implemented using a physical device integrating or installing the relevant computer program, such as a storage device. The following description uses a storage device as an example. Figure 2 As shown, the method may include the following steps.

[0116] S201, Receive a request to process the target data.

[0117] A processing request for target data can refer to a client's request to the storage device to operate on target data. The client can be a server, application, user terminal device, etc., and the target data can be the specific data that the client needs to operate on.

[0118] The processing request for the target data can be a write request, a read request, a modification request, a deletion request, etc.

[0119] Understandably, in a storage system, the essence of processing a request can be an input / output request.

[0120] The processing request may include the location information of the target data.

[0121] Location information serves as a identifier for the target data within the storage system, providing a basis for subsequent log unit allocation. Essentially, location information uniquely locates the target data. Location information can be the logical storage volume number to which the target data belongs, such as Volume 1, Volume 2, Volume 100, etc. It can also be a logical block address, i.e., the specific offset address of the target data within the storage volume; for example, it could be the 0-4MB range within Volume 1. Optionally, location information may also include data stripe numbers, storage partition identifiers, etc.

[0122] In one possible implementation, after receiving a processing request from a client, the storage device can perform a verification process on the request to check whether the client has the necessary permissions and whether the request format is correct. Only after the verification is successful will the processing request be further processed.

[0123] S202. Identify multiple log units within the storage device.

[0124] Storage devices can be physical storage servers, and can include hardware resources such as central processing units, memory, high-speed journaling hard drives, and low-speed data hard drives.

[0125] A log unit can be a log instance, meaning it can be an independent task processing unit within a storage node, used for independent task processing. Each log unit can have its own independent thread, memory partition, and dedicated log disk storage space. Furthermore, each log unit can independently receive, process, and store processing requests without resource contention with other log units. Additionally, each log unit has fault isolation capabilities; that is, a failure in one log unit does not affect the operation of other units.

[0126] In one possible implementation, multiple log units can be determined as follows: The log management module within the storage device can scan the hardware resources within the storage device, count the number of available high-speed log disks, and create a corresponding number of log units based on the number of log disks. For example, 3 log disks can correspond to 3 log units. Dedicated resources are allocated to each log unit, and each log unit is bound to an independent thread pool, a fixed memory partition is divided, and a dedicated log disk partition is specified. At the same time, a log unit list is maintained, recording the number, resource configuration, running status, etc. of all log units for subsequent steps to call.

[0127] S203. Perform a hash operation on the location information to determine the identification information corresponding to the processing request.

[0128] Hash operations refer to algorithms that convert inputs of arbitrary length into outputs of fixed length. For example, location information can be converted into identification information. Specifically, hash operations can be modulo hash operations. In essence, hash operations can distribute the outputs of different inputs evenly across multiple log units.

[0129] Identification information can be used to indicate the log unit to which the processing request is assigned, that is, the log unit number. The identification information can be the output result of a hash operation. For example, the identification information can be log unit 1, log unit 2, log unit 3, etc.

[0130] In one possible implementation, the identification information corresponding to the processing request can be determined as follows: based on the location information, determine the storage volume number corresponding to the target data; determine the number of multiple log units; and based on the number of units and the storage volume number, determine the identification information corresponding to the processing request.

[0131] A storage volume is a logical storage unit that a storage device provides to the outside world. One storage volume corresponds to one or more physical hard disk partitions.

[0132] The storage volume number can refer to the unique identifier of the storage volume. For example, the storage volume number can be volume 1, volume 2, volume 3, etc. The storage volume number can also be a core subfield of the location information.

[0133] The number of units can be the total number of log units created within the storage node. For example, the number of units can be 3, 5, 10, etc. The number of units can be a key parameter for hash operations, used to determine the value range of the identification information. For example, when the number of units is 5, the value range of the identification information is 0 to 4.

[0134] S204. Based on the identification information, determine the target log unit among multiple log units, and write the processing request to the storage space corresponding to the target log unit.

[0135] The target log unit can refer to the log unit that the identification information points to and which will ultimately process the processing request. In other words, the target log unit is the unit that uniquely matches the current processing request among multiple log units.

[0136] The storage space corresponding to the target log unit can refer to the log disk partition corresponding to the target log unit, which is the core storage carrier of the target log unit.

[0137] Writing the processing request to the storage space corresponding to the target log unit may include: the target log unit parsing the target data in the processing request, allocating free storage space in the dedicated log hard disk partition, writing the target data to the space to complete the disk write operation, updating the data status in the memory cache, marking the data as written, the target log unit returning a successful request processing response to the client, and the background flush thread asynchronously synchronizing the data in the storage space to the hard disk drive (HDD) data disk.

[0138] In the embodiments of this application, when log processing is required, a processing request for target data can be received. The processing request can refer to an operation request for target data initiated by the client to the storage device. The processing request includes the location information of the target data, which can be a location identifier of the target data in the storage system. Multiple log units are determined within the storage device. Each log unit can be a log instance, meaning it can be an independent task processing unit within the storage node, used for independent task processing. A hash operation is performed on the location information to determine the identifier information corresponding to the processing request. This identifier information can be used to indicate the log unit to which the processing request is allocated. Based on the identifier information, a target log unit is determined among the multiple log units, and the processing request is written to the storage space corresponding to the target log unit. The target log unit can refer to the log unit ultimately processing the processing request, as indicated by the identifier information. Thus, through the above method, location information can be extracted based on the processing request, and the target log unit corresponding to the processing request can be determined through hash operation, thereby enabling precise allocation. Simultaneously, the performance bottleneck problem of the traditional single log stream mode is solved through the collaboration of the hash operation algorithm and multiple independent log units.

[0139] Below, in conjunction with Figure 3 The process of determining the identification information is explained.

[0140] Figure 3 This is a schematic diagram illustrating the process of determining identification information provided in an embodiment of this application. Based on the above embodiments, see also... Figure 3 The method includes:

[0141] S301. Based on the location information, determine the storage volume number corresponding to the target data.

[0142] Location information can be carried in client processing requests to locate target data.

[0143] Location information may include the storage volume number or the logical block address.

[0144] The storage volume number can be a unique numerical identifier for the storage volume. The storage volume is the smallest logical storage unit provided by the storage device. A storage volume can correspond to a logical partition of one or more physical hard drives. The client will recognize the storage volume as a virtual hard drive.

[0145] In one possible implementation, the storage volume number can be validated to ensure its validity. Specifically, the existence of the storage volume number can be checked first. If the storage volume number exists, the location of the storage volume number can be checked. If the location is invalid, the processing request is rejected and an error message is returned to the client.

[0146] S302. Determine the number of units for multiple log units.

[0147] A log unit can refer to a log instance, and the number of units can be the number of log units that are currently available in the storage device.

[0148] In one possible implementation, the log cell status information maintained by the log management module of the storage device can be obtained. The log cells are then filtered in the log cell status information to determine the log cells in the available state, and the number of available log cells is determined as the number of cells.

[0149] S303. Determine the identification information corresponding to the processing request based on the number of units and the storage volume number.

[0150] The identification information can refer to the number of the target log unit, i.e., the output result of the hash operation.

[0151] The identification information corresponding to the processing request can be determined in the following two ways:

[0152] Method 1: In a multi-storage-volume scenario, the storage volume number is modulo-processed according to the number of units to obtain the first result of the modulo-processing; the first result is determined as the identification information corresponding to the processing request.

[0153] Modulo operation can be used to calculate the remainder, which is a commonly used hash algorithm in the storage field. The formula for modulo operation can be expressed as: First result = storage volume number % number of units.

[0154] For example, assuming the storage volume number is 5 and the number of units is 3, the first result is 2, that is, the identification information corresponding to the processing request can be considered to be 2. When the starting value of the log unit number is 0, the target log unit number can also be considered to be 1.

[0155] In this way, in a multi-volume scenario, requests for different volumes can be allocated to different log units, thereby achieving volume-level resource isolation. Furthermore, all requests for the same storage volume are allocated to the same unit, which facilitates subsequent data reconstruction, flushing, and other operations, thereby ensuring data consistency.

[0156] Method 2: In a scenario where there is only one storage volume, i.e., a single-volume scenario, obtain the preset data granularity; determine the logical block address corresponding to the target data based on the location information; and determine the identification information corresponding to the processing request based on the data granularity, logical block address, and storage volume number.

[0157] Data granularity refers to the stripe partitioning granularity in a single-volume scenario. For example, data granularity can be 4MB, 8MB, etc., and can be a value pre-configured in the storage device. Data granularity can be used to divide the storage space within a storage volume, that is, to divide it into multiple data blocks of the same size. Each data block can be assigned to a different log unit, thereby enabling parallel processing within a single volume.

[0158] A logical block address can be used to indicate the offset of target data within a storage volume, that is, the offset address of the target data within the storage volume, measured in bytes or sectors, and is used to locate the specific position of the target data within the storage volume. For example, a logical block address can be 4MB.

[0159] Assuming the logical block address is represented by LBA, the data granularity by k, the storage volume number by lunid, and the number of units by insnr, then the identification information instance id can be represented as:

[0160] instance id=((LBA / k) + lunid ) % insnr

[0161] In this way, requests for different data blocks within a single volume can be allocated to different log units, thereby enabling parallel processing within a single volume, making full use of the bandwidth of multiple log hard drives, and the data granularity can be adjusted according to the scenario to adapt to different business scenarios.

[0162] exist Figure 3 In the illustrated embodiment, when identification information needs to be determined, the storage volume number corresponding to the target data can be determined based on the location information. The location information may include the storage volume number or the logical block address. The number of multiple log units can be determined. A log unit can refer to a log instance, and the number of units can be the number of log units currently available in the storage device. Based on the number of units and the storage volume number, the identification information corresponding to the processing request can be determined. The identification information may refer to the number of the target log unit. Specifically, in a multi-volume scenario, the storage volume number can be modulo-processed based on the number of units to obtain a first result of the modulo-processing, and the first result can be determined as the identification information corresponding to the processing request. In a single-volume scenario, a preset data granularity can be obtained. Based on the location information, the logical block address corresponding to the target data can be determined, and based on the data granularity, the logical block address, and the storage volume number, the identification information corresponding to the processing request can be determined. In this way, the above method can take into account both single-volume and multi-volume scenarios, thereby improving scenario adaptability. At the same time, in the multi-volume scenario, multiple log units process multi-volume requests in parallel, and in the single-volume scenario, multiple log units process requests for different data blocks within a single volume in parallel, which can improve parallel processing capabilities and thus resolve the performance bottleneck problem of the storage system.

[0163] Below, in conjunction with Figure 4 The reconstruction process of the log unit is explained.

[0164] Figure 4 This is a schematic diagram illustrating the log unit reconstruction process provided in an embodiment of this application. Based on the above embodiments, see also... Figure 4 The method includes:

[0165] S401, Obtain rebuild instructions for multiple log units.

[0166] Rebuild commands can be generated after a storage device restarts. Understandably, when a storage device restarts after being shut down or undergoing maintenance, such as a power outage, hardware failure, or software malfunction, the memory data of the log units is lost after the restart. The data needs to be recovered from the log disk, that is, the data in the log units needs to be recovered and rebuilt. At this time, rebuild commands will be sent to each log unit to rebuild the data in the log unit.

[0167] S402. Obtain log information corresponding to multiple log units.

[0168] Log information can be a collection of metadata and business data stored in the dedicated log disk of each log unit. Log information can include log header information, log transaction data and status information. The log header information can include core metadata such as offset, verification information and unit number. The log transaction data can include business data written by the client, such as order and transaction data, and is stored in units of transactions. The status information can include the running status of the log unit, such as the last write time and flush progress.

[0169] Understandably, the log information of each log unit is stored independently in its bound dedicated log disk partition, and the log information of each log unit does not affect each other.

[0170] S403. Based on the log information, determine the valid range corresponding to multiple log units.

[0171] The valid range can refer to the valid data range in the log disk of the log unit that has not been persisted and needs to be rebuilt. The data in this range is the key data that the client has confirmed was successfully written, but has not yet been flushed back to the HDD data disk.

[0172] In one possible implementation, for any log unit, the valid interval corresponding to the log unit can be determined as follows: obtain the log information of the log unit; determine the first offset and the second offset within the log information; and determine the position interval between the first offset and the second offset as the valid interval corresponding to the log unit based on the first offset and the second offset.

[0173] The first offset and the second offset can be determined by the log header information in the log information, and the address of the first offset must be greater than the second offset, otherwise it is invalid data.

[0174] The first offset can be represented as the current write offset. The first offset can be used to indicate the current data write position of the log unit, that is, the offset of the last data written before the log unit crashes. The first offset can be used as the end boundary of the valid interval.

[0175] The second offset can represent the recovered offset. The second offset is used to indicate the position where the log unit recovers the data, that is, the offset of the position where the last record was completed and flushed back to the HDD data disk before the log unit crashes. The second offset can be used as the starting boundary of the valid interval.

[0176] The position interval between the first offset and the second offset can refer to the continuous address range defined by the first offset and the second offset. That is, the position interval can be represented as [second offset, first offset). It is understood that the position interval includes the second offset but does not include the first offset.

[0177] In this way, by defining the effective range by offset, only the core data within the effective range that needs to be recovered is processed, avoiding the reconstruction of irrelevant data, thereby improving reconstruction efficiency.

[0178] S404. Based on the valid intervals corresponding to multiple log units, perform parallel reconstruction processing on multiple log units.

[0179] The reconstruction process can include parsing, restoring, writing to disk, and flushing data within the valid range, and can be used to restore the data state before the crash.

[0180] Parallel reconstruction processing can refer to assigning independent reconstruction task threads to each log unit. The reconstruction process of each log unit is carried out simultaneously and without interference, which can improve reconstruction efficiency compared to serial reconstruction.

[0181] In one possible implementation, the reconstruction process for any log unit may include: reading log data within the valid range and parsing it according to the log transaction structure; verifying the integrity of each transaction, for example, by using cyclic redundancy check to filter out corrupted or truncated transactions; restoring the parsed valid transaction data to the dedicated memory cache of the log unit, i.e., restoring the memory state before the crash; rewriting the data in memory to the dedicated log disk of the log unit; triggering a background flush thread to asynchronously flush the data within the valid range to the HDD data disk, thereby completing the final persistence; updating the header information of the log information to mark the data within the valid range as having been reconstructed and reclaimable.

[0182] exist Figure 4 In the illustrated embodiment, when log units need to be rebuilt, rebuild instructions for multiple log units can be obtained. These instructions can be generated after the storage device restarts. Log information corresponding to multiple log units is obtained. This log information can be a collection of metadata and business data stored in each log unit's dedicated log hard drive. The log information may include log header information, log transaction data, and status information. Based on the log information, valid intervals corresponding to multiple log units are determined. Valid intervals refer to the valid data intervals in the log hard drive of each log unit that have not been persisted and need to be rebuilt. Based on the valid intervals corresponding to multiple log units, parallel rebuilding processing is performed on multiple log units. This rebuilding process may include parsing, restoring, writing to disk, and flushing the data within the valid intervals, which can be used to restore the data state before the crash. In this way, multiple log units can process rebuilding tasks simultaneously, improving rebuilding efficiency. Furthermore, the corresponding business is restored immediately upon completion of rebuilding a single log unit, eliminating the need for core businesses to wait for a full rebuild. Simultaneously, by defining the valid interval using offsets, only core data within the valid interval that needs to be restored is processed, avoiding the rebuilding of irrelevant data, thereby improving rebuilding efficiency. It also ensures that all data confirmed by the client for writing can be restored, improving system stability.

[0183] Below, in conjunction with Figure 5 The adjustment process for the log unit is explained.

[0184] Figure 5 This is a schematic diagram illustrating the adjustment process of the log unit provided in an embodiment of this application. Based on the above embodiments, see also... Figure 5 The method includes:

[0185] S501, Obtain status information for multiple log units.

[0186] Status information can be a set of core indicators reflecting the operational status of a log unit. It can indicate the resource usage status of the log unit. Status information can also include information such as processing capacity and request queues.

[0187] Resource usage status can refer to the resource usage of log units. For example, resource usage status can include computing thread utilization, CPU utilization, log disk utilization, input / output bandwidth utilization, cache hit rate, memory utilization, etc.

[0188] S502. Adjust multiple log units based on status information.

[0189] Adjusting log units can include merging or splitting them.

[0190] Merging log units can refer to integrating the resources and tasks of multiple log units into one log unit, thereby releasing redundant resources. Specifically, this can include migrating tasks, merging resources, and deregistering redundant units.

[0191] Splitting a log unit can refer to dividing the resources and tasks of a log unit into multiple new log units to distribute the load. Specifically, this can include allocating new resources to new log units, splitting existing logs, starting new log units, and migrating the load.

[0192] In one possible implementation, for any log unit, the log unit can be adjusted as follows: Based on status information, determine the data request frequency of multiple log units; obtain the resource utilization rate of the storage device; based on the resource utilization rate of the storage device and the data request frequency of multiple log units, determine the load parameters corresponding to multiple log units, where the load parameters indicate the load level of the log unit; for any log unit, merge the log unit if the load parameter corresponding to the log unit is less than or equal to a preset threshold; and split the log unit if the load parameter corresponding to the log unit is greater than the preset threshold.

[0193] Data request frequency refers to the number of input and output requests received by a log unit per unit of time. Data request frequency can be used to reflect business pressure, and it is positively correlated with resource utilization.

[0194] Storage device resource utilization can refer to the overall resource usage of the entire storage device. Storage device resource utilization includes total CPU utilization, total memory utilization, and total log disk I / O bandwidth utilization.

[0195] The load parameter can be a load score calculated by combining local resource usage (i.e., data request frequency) and global resource status (i.e., storage device resource utilization). The higher the load parameter, the higher the current load of the log unit. In one possible implementation, the load parameter can be a weighted sum of the storage device resource utilization and the data request frequency of multiple log units.

[0196] The preset threshold can be a value pre-set by the user and used as a critical value for the load parameter. For example, the preset threshold can be 80. Understandably, when the load parameter is less than or equal to the preset threshold, the current log unit's load is considered low. To save redundant resources, it can be merged with other log units, thus releasing redundant resources. When the load parameter is greater than the preset threshold, the current log unit's load is considered high, and the computational pressure on the log unit is large, requiring splitting to distribute the pressure.

[0197] exist Figure 5In the illustrated embodiment, when adjustments to log units are needed, status information of multiple log units can be obtained. This status information can be a set of core indicators reflecting the operational status of the log units, and can be used to indicate the resource occupancy status of the log units. Based on the status information, adjustments are made to multiple log units, including merging or splitting them. Thus, by dynamically adjusting the number of log instances to achieve elastic resource allocation, this method solves the resource idleness or overload problems caused by traditional fixed instance counts, ensuring the system is always in the optimal load range, avoiding performance fluctuations caused by single log unit overload, and preventing waste caused by resource idleness. This allows the system to maintain performance stability in both high-concurrency and low-load scenarios, improving resource utilization and business continuity. Furthermore, weighted calculations enable more accurate dynamic adjustments, solving the misjudgment problem caused by traditional single indicators. By comprehensively evaluating the load status through multi-dimensional indicators, the accuracy of dynamic adjustments is ensured.

[0198] To facilitate understanding, the following will be explained... Figure 6 The complete log processing flow is explained. Figure 6 A flowchart illustrating the complete log processing method provided in the application embodiment is shown below. Figure 6 As shown, the method includes:

[0199] S601. Receive a processing request for target data and determine the location information based on the processing request.

[0200] S602. Identify multiple log units within the storage device.

[0201] S603. Based on the location information, determine the storage volume number or logical block address corresponding to the target data.

[0202] S604. Determine the identification information based on the storage volume number or logical block address.

[0203] S605. Based on the identification information, determine the target log unit among multiple log units and write the processing request to the storage space corresponding to the target log unit.

[0204] S606. Receive reconstruction instructions for multiple log units and obtain log information corresponding to the multiple log units.

[0205] S607. Determine the valid intervals corresponding to multiple log units, and perform parallel reconstruction processing on multiple log units based on the valid intervals.

[0206] S608. Obtain the status information of multiple log units, and adjust the multiple log units according to the status information.

[0207] The specific implementation method and technical effects in this embodiment are similar to those in the above embodiments, and will not be repeated here.

[0208] Figure 7 This is a schematic diagram of the structure of a log processing device provided in an embodiment of this application, as shown below. Figure 7 As shown, the device 70 includes: a receiving module 701, a first determining module 702, a second determining module 703, and a writing module 704.

[0209] The receiving module 701 is used to receive a processing request for target data, the processing request including the location information of the target data;

[0210] The first determining module 702 is used to determine multiple log units within the storage device, and each log unit is used to independently process tasks.

[0211] The second determining module 703 is used to perform hash operation on the location information to determine the identification information corresponding to the processing request. The identification information is used to indicate the log unit allocated to the processing request.

[0212] The write module 704 is used to determine the target log unit among multiple log units based on the identification information, and write the processing request to the storage space corresponding to the target log unit.

[0213] The apparatus in this embodiment can be used to execute the technical solutions of the above method embodiments. The specific implementation methods and technical effects are similar, and will not be described again here.

[0214] Optionally, in the above apparatus, the second determining module 703 is specifically used for,

[0215] Based on the location information, determine the storage volume number corresponding to the target data;

[0216] Determine the number of units for multiple log units;

[0217] Based on the number of units and the storage volume number, determine the identification information corresponding to the processing request.

[0218] Optionally, in the above apparatus, the second determining module 703 is specifically used for,

[0219] Based on the number of units, the storage volume number is processed by taking a modulo to obtain the first result of the modulo processing;

[0220] The first result is determined as the identification information corresponding to the processing request.

[0221] Optionally, in the above apparatus, the second determining module 703 is specifically used for,

[0222] When the number of storage volumes is 1, obtain the preset data granularity. The data granularity is used to divide the storage space within the storage volume.

[0223] Based on the location information, the logical block address corresponding to the target data is determined. The logical block address is used to indicate the offset of the target data within the storage volume.

[0224] The identification information corresponding to the processing request is determined based on the data granularity, logical block address, and storage volume number.

[0225] The apparatus in this embodiment can be used to execute the technical solutions of the above method embodiments. The specific implementation methods and technical effects are similar, and will not be described again here.

[0226] Figure 8 This is a schematic diagram of another log processing device provided in an embodiment of this application. Figure 7 Based on what is shown, as Figure 8 As shown, the device also includes a reconstruction processing module 705 and an adjustment module 706.

[0227] Optionally, in the above-described apparatus, the reconstruction processing module 705 is used for,

[0228] Receive rebuild instructions for multiple log units; the rebuild instructions are generated after the storage device restarts.

[0229] Retrieve log information corresponding to multiple log units;

[0230] Based on the log information, determine the valid range corresponding to multiple log units;

[0231] Based on the valid intervals corresponding to multiple log units, multiple log units are reconstructed in parallel.

[0232] Optionally, in the above-described apparatus, the reconstruction processing module 705 is specifically used for,

[0233] Retrieve log information from log units;

[0234] Within the log information, determine the first offset and the second offset. The first offset is used to indicate the current data writing position of the log unit, and the second offset is used to indicate the position where the log unit reclaims the data.

[0235] Based on the first offset and the second offset, the position interval between the first offset and the second offset is determined as the valid interval corresponding to the log unit.

[0236] Optionally, in the above-described device, the adjustment module 706 is used for,

[0237] Obtain status information for multiple log units; the status information is used to indicate the resource occupancy status of the log units.

[0238] Adjustments are made to multiple log units based on status information.

[0239] Optionally, in the above-described apparatus, the adjustment module 706 is specifically used for,

[0240] Based on the status information, determine the data request frequency of multiple log units;

[0241] Obtain the resource utilization rate of storage devices;

[0242] Based on the resource utilization of the storage device and the data request frequency of multiple log units, determine the load parameters corresponding to multiple log units. The load parameters are used to indicate the load level of the log units.

[0243] For any log unit, if the load parameter corresponding to the log unit is less than or equal to a preset threshold, the log unit is merged.

[0244] When the load parameter corresponding to a log unit exceeds a preset threshold, the log unit is split.

[0245] The apparatus in this embodiment can be used to execute the technical solutions of the above method embodiments. The specific implementation methods and technical effects are similar, and will not be described again here.

[0246] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the electronic device 90 may include at least one processor 901 and a memory 902.

[0247] The memory 902 is used to store programs. Specifically, the program may include program code, which includes computer-executable instructions.

[0248] The memory 902 may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage.

[0249] The processor 901 is used to execute computer execution instructions stored in the memory 902 to implement the method described in the foregoing method embodiments. The processor 901 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0250] Optionally, the electronic device 90 may also include a communication interface 903. In specific implementations, if the communication interface 903, memory 902, and processor 901 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0251] Optionally, in a specific implementation, if the communication interface 903, memory 902, and processor 901 are integrated on a single chip, then the communication interface 903, memory 902, and processor 901 can communicate through an internal interface.

[0252] Electronic devices 90 can be servers, etc.

[0253] The electronic device in this embodiment can be used to execute the technical solutions of the above method embodiments. The specific implementation methods and technical effects are similar, and will not be repeated here.

[0254] This application provides a computer-readable storage medium, which may include various media capable of storing computer-executable instructions, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), RAM, a disk, or an optical disk. Specifically, the computer-readable storage medium stores computer-executable instructions, which, when executed by a computer, cause the technical solution shown in the above method embodiment to be executed. The specific implementation and technical effects are similar and will not be repeated here.

[0255] This application provides a computer program product, including a computer program. When the computer program is executed by a computer, the technical solution shown in the above method embodiment is executed. The specific implementation method and technical effect are similar, and will not be repeated here.

[0256] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0257] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0258] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0259] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0260] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0261] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0262] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification.

[0263] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0264] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A log processing method, characterized in that, include: Receive a processing request for target data, the processing request including the location information of the target data; Multiple log units are defined within the storage device, each log unit being used for independent task processing; The location information is hashed to determine the identification information corresponding to the processing request. The identification information is used to indicate the log unit allocated to the processing request. Based on the identification information, a target log unit is determined among the plurality of log units, and the processing request is written into the storage space corresponding to the target log unit.

2. The method according to claim 1, characterized in that, The location information is hashed to determine the identifier information corresponding to the processing request, including: Based on the location information, determine the storage volume number corresponding to the target data; Determine the number of units in the plurality of log units; The identification information corresponding to the processing request is determined based on the number of units and the number of the storage volume.

3. The method according to claim 2, characterized in that, Based on the number of units and the storage volume number, determine the identification information corresponding to the processing request, including: Based on the number of units, the number of the storage volume is modulo-processed to obtain the first result of the modulo-processing; The first result is determined as the identification information corresponding to the processing request.

4. The method according to claim 3, characterized in that, Based on the number of units and the number of the storage volume, the identification information corresponding to the processing request is determined, and the method further includes: When the number of storage volumes is 1, a preset data granularity is obtained, which is used to divide the storage space within the storage volume; Based on the location information, the logical block address corresponding to the target data is determined, and the logical block address is used to indicate the offset of the target data within the storage volume; The identification information corresponding to the processing request is determined based on the data granularity, the logical block address, and the storage volume number.

5. The method according to any one of claims 1-4, characterized in that, After writing the processing request to the storage space corresponding to the target log unit, the method further includes: Receive reconstruction instructions for the plurality of log units, wherein the reconstruction instructions are generated after the storage device is restarted; Obtain the log information corresponding to the multiple log units; Based on the log information, determine the valid intervals corresponding to the plurality of log units; Based on the valid intervals corresponding to the multiple log units, the multiple log units are reconstructed in parallel.

6. The method according to claim 5, characterized in that, Based on the log information, for any given log unit, determine the valid interval corresponding to the plurality of log units, including: Obtain the log information of the log unit; Within the log information, a first offset and a second offset are determined. The first offset is used to indicate the current data writing position of the log unit, and the second offset is used to indicate the position where the log unit reclaims the data. Based on the first offset and the second offset, the position interval between the first offset and the second offset is determined as the valid interval corresponding to the log unit.

7. The method according to any one of claims 1-4, characterized in that, After writing the processing request to the storage space corresponding to the target log unit, the method further includes: Obtain the status information of the plurality of log units, wherein the status information is used to indicate the resource occupancy status of the log units; The multiple log units are adjusted based on the status information.

8. The method according to claim 7, characterized in that, Based on the status information, adjustments are made to the plurality of log units, including: Based on the status information, determine the data request frequency of the plurality of log units; Obtain the resource utilization rate of the storage device; Based on the resource utilization of the storage device and the data request frequency of the multiple log units, the load parameters corresponding to the multiple log units are determined, and the load parameters are used to indicate the load level of the log units; For any log unit, if the load parameter corresponding to the log unit is less than or equal to a preset threshold, the log unit is merged. When the load parameter corresponding to the log unit is greater than the preset threshold, the log unit is split.

9. A log processing device, characterized in that, include: The receiving module is configured to receive a processing request for target data, the processing request including the location information of the target data; The first determining module is used to determine multiple log units within the storage device, wherein the log units are used to independently process tasks; The second determining module is used to perform a hash operation on the location information to determine the identification information corresponding to the processing request, and the identification information is used to indicate the log unit allocated to the processing request; The writing module is used to determine the target log unit among the plurality of log units based on the identification information, and write the processing request into the storage space corresponding to the target log unit.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images