A method, device, equipment and computer-readable storage medium for storing logs

Abstract

The present invention discloses a log disk-falling method, which comprises the following steps: when a log disk-falling request is received, an mds lock is occupied; the log disk-falling request is parsed to obtain the target log segment to be dropped and each index node corresponding to the target log segment; backtrack information of each index node is obtained respectively; the target log segment is dropped, and each backtrack information is sent to a preset message queue; the mds lock is released, and each backtrack information is packaged and dropped using a preset message queue and a preset thread pool. The log disk-falling method provided by the present invention improves the efficiency of log disk-falling, greatly reduces the delay of log disk-falling, and reduces the impact on other services of the metadata server. The present invention also discloses a log disk-falling device, equipment and storage medium, which have corresponding technical effects.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a log storage method, device, equipment and computer-readable storage medium. Background Art

[0002] In the current distributed file system, all operations of the metadata server (MDS) are recorded in the journal segment, which records log events. Each metadata operation is recorded as a log event. In addition to packing and storing the dentry information included in the log event itself to mark whether the metadata needs to be packed and stored on the disk, the metadata server also needs to pack and store the backtrace information of the inode involved in the log event for log backtracking.

[0003] The metadata server will periodically initiate a tick operation, occupy the mds lock to process the log segments that need to be written to the disk, and block other operations of the process. For the log segments that need to be written to the disk, the backtarce information is packaged in units of the inodes involved in this log segment, and the packaging operation is performed in sequence. The entire process is single-threaded. If there are too many inodes involved, or if one or several packaging operations experience latency fluctuations during the packaging process, the entire log segment disk operation will cause a large latency fluctuation. When the metadata server is performing a log segment disk operation, all business threads of the metadata server will be suspended. Only when the current log segment disk task is completed can other business threads continue. The latency fluctuation of the current log disk operation thread is large, which will further cause latency jams in other metadata server services.

[0004] In summary, how to effectively solve the problem that the latency fluctuation of the operation thread of the existing log storage method is large, which easily causes latency and lag in other services of the metadata server, is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] The purpose of the present invention is to provide a log disk write method, which improves the log disk write efficiency, greatly reduces the log disk write delay, and reduces the impact on other services of the metadata server; another purpose of the present invention is to provide a log disk write device, equipment and computer-readable storage medium.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A method for storing logs on a disk, comprising:

[0008] When receiving a log write request, the MDS lock is occupied;

[0009] Parsing the log storage request to obtain the target log segment to be stored and the index nodes corresponding to the target log segment;

[0010] Obtaining backtrack information of each index node respectively;

[0011] Perform a disk operation on the target log segment and send each backtrack information to a preset message queue;

[0012] The mds lock is released, and the backtrack information is packaged and stored on disk using the preset message queue and the preset thread pool.

[0013] In a specific implementation of the present invention, the preset message queue and the preset thread pool are used to package and store each backtrack information into a disk, including:

[0014] Calling the message queue processing thread of the preset message queue to send each backtrack information to the preset data pool;

[0015] Determine whether all the backtrack information has been sent out;

[0016] If yes, then wake up each packaging and disk-storage thread in the preset thread pool;

[0017] Allocate each backtrack information in the preset data pool to each packaging and disk-dropping thread;

[0018] Each of the packing and disk-storing threads is used to pack and disk-storage each of the backtrack information.

[0019] In a specific implementation of the present invention, determining whether all the backtrack information has been sent out includes:

[0020] Obtaining a data operation status flag of the preset message queue;

[0021] Determine whether the data operation status flag is an information sending completion flag.

[0022] In a specific implementation of the present invention, before the message queue processing thread of the preset message queue is called to send each backtrack information to the preset data pool, it also includes:

[0023] Setting a data receiving lock for the preset data pool to lock the preset data pool into a data receiving state;

[0024] When it is determined that all the backtrack information has been sent, the method further includes:

[0025] A cancel operation is performed on the data receiving lock to release the preset data pool.

[0026] In a specific implementation of the present invention, before allocating each backtrack information in the preset data pool to each packaging and disk-dropping thread, the method further includes:

[0027] Setting a data distribution lock for the preset data pool to lock the preset data pool into a data distribution state;

[0028] After allocating each backtrack information in the preset data pool to each packaging and disk-dropping thread, the method further includes:

[0029] The data distribution lock is canceled to release the preset data pool.

[0030] In a specific implementation of the present invention, allocating each backtrack information in the preset data pool to each packaging and disk-dropping thread includes:

[0031] Obtain the number of backtrack information to be processed corresponding to each of the packaging and disking threads;

[0032] Determine the thread heat corresponding to each of the packaging and disking threads according to the number of backtrack information to be processed; wherein the thread heat is proportional to the number of backtrack information to be processed;

[0033] Determine the allocation priority corresponding to each of the packaging and disk threads according to the heat of each of the threads;

[0034] Allocate each backtrack information in the preset data pool to each packaging and disk-dropping thread according to each allocation priority.

[0035] In a specific implementation of the present invention, determining the allocation priority corresponding to each of the packaging and disk threads according to the heat of each of the threads includes:

[0036] Determine whether there are packaging and disk-storage threads with the same thread heat;

[0037] If so, the thread numbers of the packaging and disking threads with the same thread heat are obtained respectively;

[0038] The allocation priorities corresponding to the packaging and disk-storage threads are determined in combination with the thread numbers and the thread heats.

[0039] A log disk placement device, comprising:

[0040] The lock occupation module is used to occupy the MDS lock when receiving a log storage request;

[0041] A request parsing module, used to parse the log storage request to obtain a target log segment to be stored and index nodes corresponding to the target log segment;

[0042] An information acquisition module, used to respectively acquire backtrack information of each of the index nodes;

[0043] A log storage module is used to store the target log segment on disk and send each backtrack information to a preset message queue;

[0044] The information packaging and disk storage module is used to release the mds lock and use the preset message queue and preset thread pool to package and store each backtrack information on the disk.

[0045] A log storage device, comprising:

[0046] Memory for storing computer programs;

[0047] The processor is used to implement the steps of the log storage method as described above when executing the computer program.

[0048] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the log storage method described above are implemented.

[0049] The log storage method provided by the present invention occupies the mds lock when receiving a log storage request; parses the log storage request to obtain the target log segment to be stored and each index node corresponding to the target log segment; obtains the backtrack information of each index node respectively; stores the target log segment on the disk, and sends each backtrack information to a preset message queue; releases the mds lock, and uses the preset message queue and the preset thread pool to package and store each backtrack information on the disk.

[0050] It can be seen from the above technical solution that when the target log segment needs to be written to disk, the mds lock is first occupied, and then the target log segment is written to disk directly, and after the backtrack information of each index node corresponding to the target log segment is obtained, the mds lock is released. The message queue and thread pool for writing the backtrack information to disk are pre-set, and after the backtrack information of each index node is obtained, the preset message queue and preset thread pool are used to package and write each backtrack information to disk. Thus, the asynchronous packaging and writing operation of each backtrack information is completed, and there is no need to wait for the backtrack information to be written to disk before releasing the mds lock, which improves the efficiency of log writing to disk, greatly reduces the delay of log writing to disk, and reduces the impact on other services of the metadata server.

[0051] Correspondingly, the present invention also provides a log disk-placing device, equipment and computer-readable storage medium corresponding to the above-mentioned log disk-placing method, which have the above-mentioned technical effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 This is a flow chart of an implementation of the log storage method in an embodiment of the present invention;

[0054] Figure 2 Another implementation flow chart of the log storage method in the embodiment of the present invention;

[0055] Figure 3 This is a structural block diagram of a log disk placement device in an embodiment of the present invention;

[0056] Figure 4 This is a structural block diagram of a log storage device in an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of the specific structure of a log storage device provided in this embodiment. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0059] See also Figure 1 , Figure 1 This is a flow chart of an implementation of a method for storing logs on a disk in an embodiment of the present invention. The method may include:

[0060] S101: When a log write request is received, the mds lock is occupied.

[0061] When the generated log segment needs to be written to disk, a log writing request is sent to the metadata server. When the metadata server receives the log writing request, it occupies the mds lock, thereby locking the metadata server thread to the log writing operation link.

[0062] S102: Parse the log write request to obtain the target log segment to be written to the disk and the index nodes corresponding to the target log segment.

[0063] The log storage request includes the target log segment to be stored and the index nodes corresponding to the target log segment. After receiving the log storage request, the metadata server parses the log storage request to obtain the target log segment to be stored and the index nodes corresponding to the target log segment.

[0064] S103: Obtain backtrack information of each index node respectively.

[0065] After parsing and obtaining the index nodes corresponding to the target log segment, the backtrack information of each index node is obtained respectively.

[0066] S104: The target log segment is stored on disk, and each backtracking information is sent to a preset message queue.

[0067] After parsing and obtaining the target log segment to be written to disk, and obtaining the backtrack information of each index node, the target log segment is written to disk. A message queue for caching each backtrack information is pre-set, and each backtrack information is sent to the preset message queue.

[0068] S105: releasing the mds lock, and using the preset message queue and the preset thread pool to package and write each backtrack information to the disk.

[0069] After the target log segment is written to disk and each backtrack information is sent to the preset message queue, the mds lock is released, and then the preset message queue and preset thread pool are used to package and write each backtrack information to disk, thereby releasing the mds lock in advance before the backtrack information is packaged and written to disk, improving the efficiency of log writing to disk, greatly reducing the delay of log writing to disk, and reducing the impact on other services of the metadata server.

[0070] It can be seen from the above technical solution that when the target log segment needs to be written to disk, the mds lock is first occupied, and then the target log segment is written to disk directly, and after the backtrack information of each index node corresponding to the target log segment is obtained, the mds lock is released. The message queue and thread pool for writing the backtrack information to disk are pre-set, and after the backtrack information of each index node is obtained, the preset message queue and preset thread pool are used to package and write each backtrack information to disk. Thus, the asynchronous packaging and writing operation of each backtrack information is completed, and there is no need to wait for the backtrack information to be written to disk before releasing the mds lock, which improves the efficiency of log writing to disk, greatly reduces the delay of log writing to disk, and reduces the impact on other services of the metadata server.

[0071] It should be noted that, based on the above embodiment, the embodiment of the present invention also provides corresponding improved solutions. In the subsequent embodiments, the same steps or corresponding steps as those in the above embodiment can be referenced to each other, and the corresponding beneficial effects can also be referenced to each other, which will not be repeated one by one in the following improved embodiments.

[0072] See also Figure 2 , Figure 2 This is another implementation flow chart of the log storage method in an embodiment of the present invention. The method may include the following steps:

[0073] S201: When a log write request is received, the mds lock is occupied.

[0074] S202: Parse the log write request to obtain the target log segment to be written to the disk and the index nodes corresponding to the target log segment.

[0075] S203: Obtain backtrack information of each index node respectively.

[0076] S204: The target log segment is stored on disk, and each backtracking information is sent to a preset message queue.

[0077] S205: Release the mds lock.

[0078] S206: Setting a data receiving lock for the preset data pool to lock the preset data pool into a data receiving state.

[0079] Before sending each backtrack information in the preset message queue to the preset data pool, a data receiving lock is set for the preset data pool, thereby locking the preset data pool to a data receiving state, avoiding the preset data pool from receiving and retrieving data at the same time, facilitating effective statistics of the data pool data and ensuring data integrity.

[0080] S207: calling the message queue processing thread of the preset message queue to send each backtrack information to the preset data pool.

[0081] After setting a data receiving lock for the preset data pool, the message queue processing thread of the preset message queue is called to send each backtrack information to the preset data pool, thereby realizing the orderly sending of the backtrack information from the preset message queue to the preset data pool.

[0082] S208: When it is determined that all backtrack information has been sent out, the data receiving lock is canceled to release the preset data pool.

[0083] When it is determined that all backtrack information has been sent to the preset data pool, the data receiving lock is canceled to release the preset data pool.

[0084] S209: Obtain the data operation status flag of the preset message queue.

[0085] According to the operation status of sending each backtrack information to the preset data pool by the message queue processing thread of the preset message queue, the corresponding data operation status flag is set, and the data operation status flag of the preset message queue is obtained. The data operation status flag includes the information sending flag and the information sending completion flag.

[0086] S210: Determine whether the data operation status flag is an information sending completion flag, if so, execute step S211, if not, do nothing.

[0087] After obtaining the data operation status flag of the preset message queue, determine whether the data operation status flag is the information sending completion flag. If so, it means that the subsequent backtarce information packaging and disk placement workflow can be started, and execute step S211. If not, it means that the subsequent backtarce information packaging and disk placement workflow cannot be started and no processing is performed.

[0088] S211: Wake up each packaging and disk-storage thread in the preset thread pool.

[0089] A thread pool containing multiple packaging and disking threads is preset, and the default state of each packaging and disking thread in the preset thread pool is a dormant state. When the data operation state flag is the information delivery completion flag, each packaging and disking thread in the preset thread pool is awakened.

[0090] S212: Setting a data distribution lock for the preset data pool to lock the preset data pool into a data distribution state.

[0091] After waking up each packaging and disk-storage thread in the preset thread pool, set a data distribution lock for the preset data pool and lock the preset data pool to the data distribution state. This prevents the preset data pool from receiving and retrieving data at the same time, and ensures that the backtrack information corresponding to each log segment is packaged and stored in order.

[0092] S213: Obtain the number of backtrack information to be processed corresponding to each packaging and disk-storage thread.

[0093] Each packaging and disk-storage thread may currently have backtarce information that is being packaged and stored on disk, and the number of backtarce information to be processed corresponding to each packaging and disk-storage thread is obtained.

[0094] S214: Determine the thread heat corresponding to each packaging and disk-storage thread according to the number of backtrack information to be processed.

[0095] Among them, the thread heat is proportional to the number of backtrack information to be processed.

[0096] The thread heat is preset to represent the busyness of the packaging and disking thread, and the thread heat is set to be proportional to the number of backtarce information to be processed. The thread heat corresponding to each packaging and disking thread is determined according to the number of backtarce information to be processed.

[0097] S215: Determine the allocation priority corresponding to each packaging and disk-storage thread according to the heat of each thread.

[0098] After determining the thread heat corresponding to each packaging and disking thread, the allocation priority corresponding to each packaging and disking thread is determined according to the thread heat. For example, the thread heat and the allocation priority can be set inversely proportional, that is, the higher the thread heat, the lower the allocation priority, so that a relatively busy packaging and disking thread is allocated relatively less backtarce information to be processed, and the task volume of each packaging and disking thread is balanced as much as possible.

[0099] In a specific implementation of the present invention, step S215 may include the following steps:

[0100] Step 1: Determine whether there are threads with the same thread heat and packaged to disk. If yes, execute step 2; if not, execute step 4.

[0101] Step 2: Obtain thread numbers of each packaged disk thread with the same thread heat;

[0102] Step 3: Determine the allocation priority corresponding to each packaging and disk thread based on the thread number and thread heat;

[0103] Step 4: Determine the allocation priority corresponding to each packaging and disk thread according to the heat of each thread.

[0104] For the convenience of description, the above four steps can be combined for explanation.

[0105] Thread numbers are set in advance for each packaged disk thread in the preset thread pool, such as 0, 1, 2, etc. for each packaged disk thread. It can be pre-set that when there are packaged disk threads with the same thread heat, the lower the number, the higher the allocation priority. After determining the thread heat corresponding to each packaged disk thread, it is determined whether there are packaged disk threads with the same thread heat. If so, the thread numbers of each packaged disk thread with the same thread heat are obtained respectively, and the allocation priorities corresponding to each packaged disk thread are determined in combination with each thread number and each thread heat. That is, the preliminary allocation priority order is first determined according to the thread heat, and then the packaged disk threads with the same thread heat are sorted according to their thread numbers to obtain the final allocation priority order. If not, the allocation priorities corresponding to each packaged disk thread are directly determined according to each thread heat. By combining each thread number and each thread heat to determine the allocation priorities corresponding to each packaged disk thread, the allocation priorities of packaged disk threads with the same thread heat are determined.

[0106] S216: Allocate each backtrack information in the preset data pool to each packaging and disk-dropping thread according to each allocation priority.

[0107] After determining the allocation priorities corresponding to the packaging and disking threads, the backtracking information in the preset data pool is allocated to the packaging and disking threads according to the allocation priorities, thereby achieving a reasonable allocation of the backtracking information packaging and disking tasks.

[0108] S217: cancel the data distribution lock to release the preset data pool.

[0109] After all backtrack information in the preset data pool is distributed to each packaging and disk-dropping thread, the data distribution lock is canceled to release the preset data pool.

[0110] S218: using each packing and disking thread to pack and disk each backtrack information.

[0111] After allocating each backtarce information in the preset data pool to each packaging and disking thread according to each allocation priority, each packaging and disking thread is used to package and disk each backtarce information, thereby realizing asynchronous multi-threaded packaging and disking of each backtarce information. After the packaging and disking of each backtarce information is completed, the thread heat of each packaging and disking thread is reset to zero, and each packaging and disking thread in the preset thread pool resumes the thread dormant state. This greatly improves the efficiency of packaging and disking of backtarce information, reduces the processing time of a single log segment, improves the efficiency of log disking, greatly reduces the delay of log disking, and reduces the impact on other services of the metadata server.

[0112] Corresponding to the above method embodiment, the present invention further provides a log-dropping device. The log-dropping device described below and the log-dropping method described above can refer to each other.

[0113] See also Figure 3 , Figure 3 This is a structural block diagram of a log storage device in an embodiment of the present invention. The device may include:

[0114] The lock occupation module 31 is used to occupy the mds lock when receiving a log storage request;

[0115] The request parsing module 32 is used to parse the log storage request to obtain the target log segment to be stored and the index nodes corresponding to the target log segment;

[0116] The information acquisition module 33 is used to respectively acquire the backtrack information of each index node;

[0117] The log storage module 34 is used to store the target log segment on the disk and send each backtrack information to the preset message queue;

[0118] The information packaging and disk storage module 35 is used to release the mds lock and use the preset message queue and the preset thread pool to package and store each backtrack information on the disk.

[0119] It can be seen from the above technical solution that when the target log segment needs to be written to disk, the mds lock is first occupied, and then the target log segment is written to disk directly, and after the backtrack information of each index node corresponding to the target log segment is obtained, the mds lock is released. The message queue and thread pool for writing the backtrack information to disk are pre-set, and after the backtrack information of each index node is obtained, the preset message queue and preset thread pool are used to package and write each backtrack information to disk. Thus, the asynchronous packaging and writing operation of each backtrack information is completed, and there is no need to wait for the backtrack information to be written to disk before releasing the mds lock, which improves the efficiency of log writing to disk, greatly reduces the delay of log writing to disk, and reduces the impact on other services of the metadata server.

[0120] In a specific embodiment of the present invention, the information packaging and disk storage module 35 includes:

[0121] The information sending submodule is used to call the message queue processing thread of the preset message queue to send each backtrack information to the preset data pool;

[0122] The judgment submodule is used to judge whether all backtrack information has been sent;

[0123] The thread wake-up submodule is used to wake up each packaging and disk-dropping thread in the preset thread pool when it is determined that all backtrack information has been sent out;

[0124] The information distribution submodule is used to distribute each backtrack information in the preset data pool to each packaging and disk-dropping thread;

[0125] The information packaging and disk placement module is used to use each packaging and disk placement thread to package and disk each backtrack information.

[0126] In a specific embodiment of the present invention, the judgment submodule includes:

[0127] An operation status flag obtaining unit, used to obtain a data operation status flag of a preset message queue;

[0128] The judging unit is used to judge whether the data operation status mark is an information sending completion mark.

[0129] In a specific embodiment of the present invention, the device may further include:

[0130] The data receiving state locking module is used to set a data receiving lock for the preset data pool before the message queue processing thread of the preset message queue is called to send each backtrack information to the preset data pool, so as to lock the preset data pool to the data receiving state;

[0131] The data receiving lock canceling module is used to cancel the data receiving lock when it is determined that all backtrack information has been sent out, so as to release the preset data pool.

[0132] In a specific embodiment of the present invention, the device may further include:

[0133] The data distribution state locking module is used to set a data distribution lock for the preset data pool before allocating each backtrack information in the preset data pool to each packaging and disking thread, so as to lock the preset data pool into the data distribution state;

[0134] The data distribution lock cancellation module is used to cancel the data distribution lock after allocating each backtrack information in the preset data pool to each packaging and disk-dropping thread, so as to release the preset data pool.

[0135] In a specific embodiment of the present invention, the information distribution submodule includes:

[0136] An information number acquisition unit is used to acquire the number of backtrack information to be processed corresponding to each packaging and disking thread;

[0137] A thread heat determination unit is used to determine the thread heat corresponding to each packaged disk thread according to the number of backtrack information to be processed; wherein the thread heat is proportional to the number of backtrack information to be processed;

[0138] An allocation priority determination unit, used to determine the allocation priority corresponding to each packaging and disking thread according to the heat of each thread;

[0139] The information distribution unit is used to distribute each backtrack information in the preset data pool to each packaging and disk-dropping thread according to each distribution priority.

[0140] In a specific implementation of the present invention, the allocation priority determination unit includes:

[0141] A judgment subunit is used to judge whether there are packaging and disk-storage threads with the same thread heat;

[0142] The thread number acquisition subunit is used to respectively acquire the thread numbers of the packaging and disking threads with the same thread heat when it is determined that there are packaging and disking threads with the same thread heat;

[0143] The allocation priority determination subunit is used to determine the allocation priority corresponding to each packaging and disk-storage thread in combination with each thread number and each thread heat.

[0144] Corresponding to the above method embodiment, see Figure 4 , Figure 4 This is a schematic diagram of the log storage device provided by the present invention, and the device may include:

[0145] A memory 332, for storing computer programs;

[0146] The processor 322 is used to implement the steps of the log storage method of the above method embodiment when executing the computer program.

[0147] For details, please refer to Figure 5 , Figure 5 A specific structural diagram of a log disk device provided for this embodiment, the log disk device may have relatively large differences due to different configurations or performances, and may include a processor (central processing units, CPU) 322 (for example, one or more processors) and a memory 332, and the memory 332 stores one or more computer applications 342 or data 344. Among them, the memory 332 can be a temporary storage or a permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the processor 322 can be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the log disk device 301.

[0148] The log storage device 301 may further include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input and output interfaces 358 , and / or one or more operating systems 341 .

[0149] The steps in the log-to-disk method described above can be implemented by the structure of the log-to-disk device.

[0150] Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps can be implemented:

[0151] When a log storage request is received, the mds lock is occupied; the log storage request is parsed to obtain the target log segment to be stored and the index nodes corresponding to the target log segment; the backtarce information of each index node is obtained respectively; the target log segment is stored on disk, and each backtarce information is sent to the preset message queue; the mds lock is released, and the preset message queue and preset thread pool are used to package and store each backtarce information on disk.

[0152] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0153] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiment, and the present invention will not be elaborated here.

[0154] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the devices, equipment and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0155] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the technical solution and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, the present invention can also be improved and modified without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for storing logs on a disk, characterized in that: include: When receiving a log write request, the MDS lock is occupied; Parsing the log storage request to obtain the target log segment to be stored and the index nodes corresponding to the target log segment; Obtaining backtrack information of each index node respectively; Perform a disk operation on the target log segment and send each backtrack information to a preset message queue; The mds lock is released, and the message queue processing thread of the preset message queue is called to send each backtrack information to the preset data pool; Determine whether all the backtrack information has been sent out; If yes, then wake up each packaging and disk-storage thread in the preset thread pool; Allocate each backtrack information in the preset data pool to each packaging and disk-dropping thread; Each of the packing and disk-storing threads is used to pack and disk-storage each of the backtrack information.

2. The log storage method according to claim 1, characterized in that: Determining whether all the backtrack information has been sent out includes: Obtaining a data operation status flag of the preset message queue; Determine whether the data operation status flag is an information sending completion flag.

3. The log storage method according to claim 1, characterized in that: Before calling the message queue processing thread of the preset message queue to send each backtrack information to the preset data pool, it also includes: Setting a data receiving lock for the preset data pool to lock the preset data pool into a data receiving state; When it is determined that all the backtrack information has been sent, the method further includes: A cancel operation is performed on the data receiving lock to release the preset data pool.

4. The log storage method according to claim 1, characterized in that: Before allocating each backtrack information in the preset data pool to each packaging and disk-dropping thread, the method further includes: Setting a data distribution lock for the preset data pool to lock the preset data pool into a data distribution state; After allocating each backtrack information in the preset data pool to each packaging and disk-dropping thread, the method further includes: The data distribution lock is canceled to release the preset data pool.

5. The log transfer method according to any one of claims 1 to 4, characterized in that: Allocating each backtrack information in the preset data pool to each packaging and disk-storage thread includes: Obtain the number of backtrack information to be processed corresponding to each of the packaging and disking threads; Determine the thread heat corresponding to each of the packaging and disking threads according to the number of backtrack information to be processed; wherein the thread heat is proportional to the number of backtrack information to be processed; Determine the allocation priority corresponding to each of the packaging and disk threads according to the heat of each of the threads; Allocate each backtrack information in the preset data pool to each packaging and disk-dropping thread according to each allocation priority.

6. The log storage method according to claim 5, characterized in that: Determining the allocation priorities corresponding to the packaging and disking threads according to the heat of the threads, including: Determine whether there are packaging and disk-storage threads with the same thread heat; If so, the thread numbers of the packaging and disking threads with the same thread heat are obtained respectively; The allocation priorities corresponding to the packaging and disk-storage threads are determined in combination with the thread numbers and the thread heats.

7. A log plate device, characterized in that: include: The lock occupation module is used to occupy the MDS lock when receiving a log storage request; A request parsing module, used to parse the log storage request to obtain a target log segment to be stored and index nodes corresponding to the target log segment; An information acquisition module, used to respectively acquire backtrack information of each of the index nodes; A log storage module is used to store the target log segment on disk and send each backtrack information to a preset message queue; An information packaging and disk-storage module is used to release the mds lock; The information packaging and disk storage module includes: The information sending submodule is used to call the message queue processing thread of the preset message queue to send each backtrack information to the preset data pool; The judgment submodule is used to judge whether all backtrack information has been sent; The thread wake-up submodule is used to wake up each packaging and disk-dropping thread in the preset thread pool when it is determined that all backtrack information has been sent out; The information distribution submodule is used to distribute each backtrack information in the preset data pool to each packaging and disk-dropping thread; The information packaging and disk placement module is used to use each packaging and disk placement thread to package and disk each backtrack information.

8. A log storage device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the log storage method as claimed in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the log storage method according to any one of claims 1 to 6 are implemented.

Images