Log file pre-retrieval method, device, medium and system

Abstract

The invention provides a log file pre-retrieval method and device, a medium and a system.The method comprises the steps that a log binding ID input by a client side is received, integrity verification is conducted on the log binding ID, and the log binding ID comprises a log timestamp; traversing a first timestamp of a root node of each improved Merkel tree according to the log timestamp of the log binding ID passing the integrity verification; under the condition that the first timestamp contains the log timestamp, determining the corresponding Merkel tree as a target Merkel tree; traversing the target Merkel tree until leaf nodes corresponding to the second timestamps and the same as the log timestamps are determined, and obtaining target leaf nodes; and capturing corresponding data from the database according to the target leaf node to obtain target data. The problem that in the prior art, the retrieval efficiency of a retrieval method based on the Merkel tree is low along with the increase of the data volume in the system is solved.

Description

Technical Field

[0001] This invention relates to the field of data communication technology, and more specifically, to a log file pre-retrieval method, a log file pre-retrieval device, a computer-readable storage medium, and a data storage system. Background Art

[0002] In existing technologies, as the amount of data in the system increases, the depth and number of nodes of the Merkle tree also increase significantly, resulting in an increase in the number of nodes that need to be traversed when querying specific log data, and the retrieval efficiency of Merkle tree-based retrieval methods is low. Summary of the Invention

[0003] The main objective of this application is to provide a pre-retrieval method, a pre-retrieval device, a computer-readable storage medium, and a data storage system for log files, so as to at least solve the problem that the retrieval efficiency of Merkle tree-based retrieval methods in the prior art is low as the amount of data in the system increases.

[0004] To achieve the above objectives, according to one aspect of this application, a pre-retrieval method for log files is provided, comprising: receiving a log binding ID input by a client and performing an integrity check on the log binding ID, the log binding ID including a log timestamp; traversing the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that passed the integrity check, wherein the leaf node of the improved Merkle tree includes a hash value corresponding to the log file, a timestamp, and the log binding ID, and the timestamp of each node of the improved Merkle tree includes the timestamps of all its corresponding child nodes; if the first timestamp contains the log timestamp, determining the corresponding Merkle tree as the target Merkle tree; traversing the target Merkle tree until a leaf node with a corresponding second timestamp that is the same as the log timestamp is determined, obtaining the target leaf node; and retrieving corresponding data from a database according to the target leaf node to obtain target data.

[0005] Optionally, the log binding ID further includes an integrity check based on a random code for locally repairable functionality. This check includes: extracting the random code from the log binding ID to obtain a candidate random code; extracting the encoding length of the candidate random code to obtain a target length; determining that the log binding ID integrity check fails if the target length is less than a preset length, and determining that the log binding ID integrity check passes if the target length is equal to the preset length; traversing each character of the candidate random code, determining that the log binding ID integrity check fails if a missing character exists, and determining that the log binding ID integrity check passes if no missing character exists.

[0006] Optionally, traversing the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed integrity verification includes: extracting the random code from the log binding ID that has passed integrity verification to obtain a first random code; verifying the first random code according to a first target matrix, wherein the first target matrix is ​​a verification matrix determined based on the generated first random code when generating the first random code; if the first random code passes verification, traversing the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID.

[0007] Optionally, verifying the first random code based on the first target matrix includes: transposing the first random code to obtain a fourth random code; calculating the product of the fourth random code and the first target matrix; and determining that the verification of the first random code fails if the product is not zero.

[0008] Optionally, after determining that the log binding ID integrity verification fails, the method further includes: extracting the random code of the log binding ID that failed the integrity verification to obtain a second random code; obtaining a second target matrix, wherein the second target matrix is ​​a verification matrix determined based on the generated second random code when generating the second random code; and performing a repair operation on the second random code based on the second target matrix.

[0009] Optionally, the repair operation on the second random code based on the second target matrix includes: transposing the second random code to obtain a third random code; solving for the value of the character with missing values ​​in the third random code based on the condition that the multiplication of the third random code and the second target matrix is ​​0, to obtain at least one target value; associating the target value with the corresponding character to complete the repair of the second random code.

[0010] Optionally, after traversing the target Merkle tree, the method further includes: if there is no leaf node in the target Merkle tree that is the same as the corresponding second timestamp and the log timestamp, traversing each of the improved Merkle trees according to the log binding ID until a leaf node corresponding to the log binding ID that is the same as the input log binding ID is determined, thereby obtaining the target leaf node.

[0011] According to another aspect of this application, a pre-retrieval apparatus for log files is provided. The apparatus includes: a first acquisition unit, configured to receive a log binding ID input by a client and perform integrity verification on the log binding ID, wherein the log binding ID includes a log timestamp; a first retrieval unit, configured to traverse the first timestamps of the root nodes of each improved Merkle tree based on the log timestamps of the log binding IDs that have passed integrity verification, wherein the leaf nodes of the improved Merkle tree include a hash value corresponding to the log file, a timestamp, and the log binding ID, and the timestamps of each node of the improved Merkle tree include the timestamps of all its corresponding child nodes; a determination unit, configured to determine the corresponding Merkle tree as a target Merkle tree if the first timestamp contains the log timestamp; a second retrieval unit, configured to traverse the target Merkle tree until a leaf node with a second timestamp identical to the log timestamp is determined, thereby obtaining a target leaf node; and a second acquisition unit, configured to retrieve corresponding data from a database based on the target leaf node, thereby obtaining target data.

[0012] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0013] According to another aspect of this application, a data storage system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of the methods described in the claims.

[0014] The technical solution of this application first receives the log binding ID input by the client and performs integrity verification on the log binding ID, which includes a log timestamp. Then, based on the log timestamp of the log binding ID that has passed integrity verification, the first timestamp of the root node of each improved Merkle tree is traversed. If the first timestamp contains the log timestamp, the corresponding Merkle tree is determined as the target Merkle tree. Next, the target Merkle tree is traversed until a leaf node with a second timestamp matching the log timestamp is determined, thus obtaining the target leaf node. Finally, the corresponding data is retrieved from the database based on the target leaf node to obtain the target data. This application enables the system to perform large-scale data retrieval operations by comparing the root node, internal nodes, and leaf nodes of the improved Merkle tree layer by layer using timestamps, avoiding the traversal of data blocks with mismatched timestamps. Compared to the prior art's global traversal of the Merkle tree using the log binding ID, this application reduces the amount of data traversed during the retrieval process and solves the problem of low retrieval efficiency of Merkle tree-based retrieval methods as the amount of data in the system increases. Attached Figure Description

[0015] Figure 1 A hardware structure block diagram of a mobile terminal for a log file pre-retrieval method provided in an embodiment of this application is shown.

[0016] Figure 2 A flowchart illustrating a pre-retrieval method for log files according to an embodiment of this application is shown.

[0017] Figure 3 A flowchart illustrating another log file pre-retrieval method provided according to an embodiment of this application is shown;

[0018] Figure 4 An improved Merkle tree structure according to an embodiment of this application is shown;

[0019] Figure 5 A structural block diagram of a log file pre-retrieval device provided according to an embodiment of this application is shown.

[0020] The above figures include the following reference numerals:

[0021] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] As described in the background section, the retrieval efficiency of Merkle tree-based retrieval methods in the prior art is low as the amount of data in the system increases. To solve the above technical problems, embodiments of this application provide a pre-retrieval method for log files, a pre-retrieval device for log files, a computer-readable storage medium, and a data storage system.

[0026] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0027] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a log file pre-retrieval method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0028] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the device information display method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] This embodiment provides a method for pre-retrieval of log files running on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] Figure 2 This is a flowchart of a pre-retrieval method for log files according to an embodiment of this application. For example... Figure 2 As shown, the method includes the following steps:

[0031] Step S101: Receive the log binding ID input by the client and perform integrity verification on the log binding ID, which includes the log timestamp.

[0032] The integrity check includes length verification and verification of the correctness of the random code portion. If the length of the random code portion does not meet the preset length, or if a character is missing, the integrity check will fail.

[0033] Step S102: Traverse the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed the integrity verification. The leaf node of the improved Merkle tree includes the hash value, timestamp and log binding ID corresponding to the log file. The timestamp of each node of the improved Merkle tree includes the timestamps of all its child nodes.

[0034] Specifically, based on the input log timestamp, the root node timestamps of all Merkle trees are traversed to find the root node that meets the condition, that is, the Merkle tree whose root node timestamp is greater than or equal to the input timestamp.

[0035] Step S103: If the first timestamp includes the log timestamp, the corresponding Merkle tree is determined as the target Merkle tree.

[0036] If the root node timestamp is the same as the input log timestamp, or if the range of the root node timestamp can cover the input timestamp, then the corresponding Merkle tree is marked as the target Merkle tree mentioned above, and becomes the object of the next retrieval.

[0037] Step S104: Traverse the target Merkle tree until the leaf node whose corresponding second timestamp is the same as the log timestamp is determined, and obtain the target leaf node;

[0038] The leaf nodes of the Merkle tree contain the hash value, timestamp, and log binding ID of the log file. The system starts from the root node of the target Merkle tree and traverses down the tree level until it finds a leaf node that matches the input log timestamp. During traversal, the left subtree is checked first; if the timestamp range of the left subtree does not include the input timestamp, the system moves to the right subtree, and so on, until the target leaf node is found.

[0039] Step S105: Retrieve the corresponding data from the database based on the target leaf node to obtain the target data.

[0040] Through the above embodiments, the system first receives the log binding ID input from the client and performs integrity verification on the log binding ID, which includes a log timestamp. Then, based on the log timestamp of the log binding ID that has passed integrity verification, the system traverses the first timestamp of the root node of each improved Merkle tree. If the first timestamp includes the log timestamp, the corresponding Merkle tree is identified as the target Merkle tree. Next, the target Merkle tree is traversed until the leaf node with the same second timestamp as the log timestamp is identified, thus obtaining the target leaf node. Finally, the corresponding data is retrieved from the database based on the target leaf node to obtain the target data. This application enables the system to perform layer-by-layer comparison of the root node, internal nodes, and leaf nodes of the improved Merkle tree using timestamps, thereby completing the retrieval operation of large blocks of data. This avoids the traversal operation of data blocks with mismatched timestamps. Compared to the prior art's global traversal of the Merkle tree using the log binding ID, this application reduces the amount of data traversed during the retrieval process and solves the problem of low retrieval efficiency of Merkle tree-based retrieval methods as the amount of data in the system increases.

[0041] In one optional embodiment, the log binding ID further includes an integrity check based on a random code for locally repairable functionality. This check includes: extracting the random code from the log binding ID to obtain a candidate random code; extracting the encoding length of the candidate random code to obtain a target length; determining that the log binding ID integrity check fails if the target length is less than a preset length, and determining that the log binding ID integrity check passes if the target length is equal to the preset length; traversing each character of the candidate random code, determining that the log binding ID integrity check fails if a missing character exists, and determining that the log binding ID integrity check passes if no missing character exists.

[0042] In the above embodiments, a candidate random code is obtained by extracting a partially repairable code from the log binding ID input by the user. This allows the verification process to be based on the encoding structure of the candidate random code, ensuring that the retrieval request relies not only on a simple identifier but also on a random code with error correction and self-verification functions, thus improving the reliability of the input data. By extracting the encoding length of the candidate random code and comparing it with a preset length threshold, the integrity of the input log binding ID can be quickly determined. If the length is insufficient, the verification fails, avoiding resource consumption caused by invalid or erroneous input and improving system response speed and computational efficiency. By iterating through each character of the candidate random code, it checks for missing values. If missing characters are found, the log binding ID integrity verification fails, ensuring that only complete and correct log binding IDs can enter the subsequent retrieval process, effectively preventing query errors or inaccurate results due to missing data.

[0043] Specifically, missing values ​​include blank inputs, illegal characters, or uncertain symbols.

[0044] In another alternative approach, traversing the first timestamp of each root node of the improved Merkle tree based on the log timestamp of the log binding ID that has passed integrity verification includes: extracting the random code from the log binding ID that has passed integrity verification to obtain a first random code; verifying the first random code based on a first target matrix, wherein the first target matrix is ​​a verification matrix determined based on the generated first random code when generating the first random code; and if the first random code passes verification, traversing the first timestamp of each root node of the improved Merkle tree based on the log timestamp of the log binding ID.

[0045] In the above embodiments, a first random code is first extracted from the log binding ID that has passed integrity verification. Then, the first random code is verified using a first target matrix corresponding to the generation of this first random code, i.e., a verification matrix for locally repairable codes. This ensures that the input random code strictly conforms to the encoding rules, effectively identifying potential encoding errors or illegal tampering, and improving the security and reliability of the log binding ID. The first target matrix is ​​a verification matrix dynamically generated based on the encoding structure determined when the first random code is generated. The dynamically generated verification matrix more accurately reflects the encoding characteristics of the random code, improves error detection capabilities, ensures the authenticity and integrity of the random code in the log binding ID, and prevents illegal data from passing through the pre-retrieval process. Under the premise that the random code verification passes, an improved Merkle tree root node traversal based on log timestamps is performed. Only after the first random code passes verification is the timestamp information in the log binding ID used to traverse the first timestamp of each root node of the improved Merkle tree. This achieves rapid location of the block where the target log is located, reduces invalid or erroneous traversals, and improves retrieval accuracy and efficiency. By combining dynamic verification of random codes with timestamp traversal, attacks that maliciously forge or tamper with log binding IDs are effectively resisted, ensuring the security of the retrieval system. At the same time, the system remains stable and reliable under high concurrency and large data volume scenarios, enhancing the robustness of the overall log pre-retrieval platform.

[0046] In some exemplary embodiments, verifying the first random code based on the first target matrix includes: transposing the first random code to obtain a fourth random code; calculating the product of the fourth random code and the first target matrix; and determining that the verification of the first random code fails if the product is not zero.

[0047] In the above embodiments, the first random code is transposed into a column vector to obtain the fourth random code, which is then multiplied with the first target matrix. This multiplication reflects whether the random code satisfies the linear constraint of the locally repairable code. A zero product vector means the random code fully conforms to the linear relationship specified by the check matrix. If the product is non-zero, it indicates that the random code violates the linear constraint, contains errors or incomplete information, and the verification fails. This scheme can accurately detect any illegal modifications, incorrect encodings, or data corruption in the random code, ensuring the mathematical integrity and encoding validity of the random code, and improving the security verification strength of the log binding ID. It also reduces manual intervention and repeated verification steps, increases verification speed, and further improves retrieval efficiency.

[0048] In other exemplary embodiments, after determining that the integrity verification of the log binding ID fails, the method further includes: extracting the random code of the log binding ID that failed the integrity verification to obtain a second random code; obtaining a second target matrix, wherein the second target matrix is ​​a verification matrix determined based on the generated second random code when the second random code is generated; and performing a repair operation on the second random code based on the second target matrix.

[0049] In the above embodiments, when the log binding ID fails the integrity check, the problematic random code portion of the binding ID is first precisely extracted to obtain a second random code, achieving accurate location of the abnormal data. The second target matrix is ​​the verification matrix corresponding to the generation of the second random code, which is a matrix dynamically generated based on the encoding rules of the random code. By obtaining the second target matrix, it is ensured that the repair process is strictly executed according to the original encoding constraints, making the repair operation more accurate, avoiding repair failures or errors caused by matrix mismatch, and improving the repair accuracy. By using the second target matrix to perform the repair operation on the second random code, self-correction of the input log binding ID is achieved, enabling accurate recovery of valid information and maintaining normal system operation even when faced with partially damaged or incomplete log binding IDs. This avoids interruption of the entire retrieval process or invalidation of results due to minor defects in the input data, significantly improving the robustness and fault tolerance of the system. Moreover, the repair process is completed automatically without manual intervention, saving human resources and further improving the overall retrieval efficiency.

[0050] According to some alternative embodiments of this application, the repair operation of the second random code based on the second target matrix includes: transposing the second random code to obtain a third random code; solving for the value of the character with missing value in the third random code based on the condition that the multiplication of the third random code with the second target matrix is ​​0, to obtain at least one target value; associating the target value with the corresponding character to complete the repair of the second random code.

[0051] In the above embodiments, firstly, the input second random code is transposed to obtain a third random code. By defining a system of linear equations and solving this system, the specific values ​​of all missing characters in the random code are determined, thereby completing the missing or erroneous parts of the second random code. This achieves rapid and accurate repair of the random code, avoiding manual intervention and improving repair efficiency and quality. By associating the target value obtained from the solution with the corresponding missing characters in the second random code one by one, replacing the originally missing or unknown parts, a complete random code that conforms to the encoding rules is finally generated. This effectively solves the problem of retrieval failure caused by missing or corrupted data in log binding ID input, enabling the system to automatically recover valid query identifiers, improving the system's tolerance to abnormal input and stable operation capability, reducing retrieval interruptions caused by data errors, and improving processing efficiency and user experience.

[0052] In some other exemplary embodiments of this application, after traversing the target Merkle tree, the method further includes: if there is no leaf node in the target Merkle tree that is the same as the corresponding second timestamp and the log timestamp, traversing each of the improved Merkle trees according to the log binding ID until a leaf node is determined that is the same as the input log binding ID, thereby obtaining the target leaf node.

[0053] In the above embodiments, a secondary retrieval mechanism based on log binding IDs is provided to address the anomaly where a leaf node with an exact timestamp match cannot be found in the target Merkle tree. This ensures that even if log timestamps are abnormal, erroneous, or lost, the target log can still be accurately located using the log binding ID, thus improving the system's fault tolerance. Using the complete input log binding ID as the retrieval keyword, all improved Merkle trees are traversed to accurately locate the corresponding log data in any block, reducing the risk of retrieval failures caused by timestamp errors, log delays, or anomalies. By completely matching the input log binding ID with the log binding IDs in the leaf nodes, the uniqueness and accuracy of the retrieval results are ensured, preventing incorrect matching and false detections, and guaranteeing the complete acquisition of critical log data.

[0054] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the pre-retrieval method for log files of this application will be described in detail below with reference to specific embodiments.

[0055] This embodiment relates to a specific method for pre-retrieval of log files, such as... Figure 3 As shown, it includes the following steps:

[0056] Step S1: Enter the log binding ID;

[0057] Specifically, user-input logs are bound to the search box and then passed to the backend. The ID is defined as (TS||genNum), where TS represents the timestamp of the transaction, in the format yyyyMMddHHmmss (14 characters long), representing year, month, day, hour, minute, and second respectively. genNum represents a random number generated by our rules (the codeword of the locally repairable code). Different lengths of locally repairable codes can be selected to meet the system's requirements for the number of random numbers. Once a specific locally repairable code is selected, it cannot be modified. Let's assume the selected locally repairable code length is n.

[0058] Step S2: Determine if the log binding ID is complete. If the log binding ID is complete, proceed to step S4. If the log binding ID is incomplete, proceed to steps S3 and S5.

[0059] Specifically, the determination of whether the length of the input genNum in step 1 is n and whether any bit value is uncertain is as follows: For a specific locally repairable code, the codeword length is a fixed value n. Therefore, the length of genNum can be used to first determine whether the length is n. If the length conforms to the rules and the data of each bit is certain, it is considered complete and proceeds to step 4. If the length conforms to the rules but a bit value is uncertain, repair is required, and the process proceeds to step 3.

[0060] Step S3: Repair log binding ID;

[0061] Repair the input genNum, where the missing bits of genNum are the other at most r bits in the finite field F. q A linear combination of equations H.genNum is calculated by computing the system of equations. T = 0, where H represents the parity-check matrix of the locally repairable code, genNum T This means that genNum is represented as a vector and then transposed, as shown in formulas (1), (2), and (3), to solve for the missing bits. This method ensures that log information can still be correctly obtained even when the input log binding ID is incomplete, thus improving the robustness and efficiency of the pre-retrieval system.

[0062] Step S4: Determine whether the log binding ID meets the generation rules. If the log binding ID meets the generation rules, proceed to step S5. If the log binding ID does not meet the generation rules, end the process.

[0063] Specifically, the input genNum is examined, and the system of equations H.genNum is calculated according to the definition of locally repairable codes. T Does = 0 hold true, where H represents the parity check matrix of the locally repairable code, and genNum TThis means that genNum is represented as a vector and then transposed, as shown in formulas (1), (2), and (3). If true, genNum is determined to conform to the generation rule, and the process proceeds to the next step. If false, an error is reported, the query is interrupted, and an empty result is returned. This method can determine whether an ID meets the requirements through formula verification without requesting blockchain data, further improving the efficiency of pre-retrieval.

[0064] Step S5: Obtain key log data;

[0065] Specifically, log pre-retrieval is performed based on the log binding ID input in the previous step. First, the TS timestamp from the ID is obtained, and then the root node in the block is used as the reference point. Figure 4 The root node shown determines whether the conditions TS≤LogTime4 and TS≥LogTime1 are met. Figure 4 If LogTime1 ≤ LogTime2 ≤ LogTime3 ≤ LogTime4, the query continues in that block; otherwise, iterates through other blocks until the corresponding block is found. Next, it checks if TS is less than the maximum timestamp value of the left subtree. If so, it continues querying the left subtree; otherwise, it continues querying the right subtree, and so on, until a leaf node is found. Then, it checks if the log binding ID stored in the leaf node is equal to the input log binding ID. If they are the same, it returns the key log data in the node; otherwise, it returns empty. Based on the characteristic of logs being stored in timestamp order, this method can more quickly locate and retrieve log information, further improving the retrieval efficiency of the pre-retrieval system.

[0066] Formula (1): Let Hg e in formula (1) T =n0, thus obtaining formula (2):

[0067] Formula (3) can be obtained from formulas (1) and (2):

[0068] In some exemplary embodiments, upon receiving a log file sent by a client, a timestamp is generated based on the log file's write time to obtain a third timestamp; a random code of a preset length is randomly generated to obtain a fifth random code; the fifth random code is transposed to obtain a sixth random code; a matrix whose product with the sixth random code is 0 is constructed to obtain the check matrix corresponding to the fifth random code; the third timestamp and the fifth random code are concatenated to obtain the log binding ID of the log file; and the log binding ID, the check matrix, and the log file are stored in the database. When the log file is stored in the database, the log file's summary information is retrieved. The summary information includes the input data, output data, error records, operation execution results, server address, and log binding ID recorded in the log file; a hash operation is performed on the summary information to obtain a target hash value; the current Merkle tree is traversed based on the third timestamp of the target hash value to determine the internal node containing the third timestamp, thus obtaining the target node; the third timestamp and the target hash value are written under the target node to obtain the leaf node corresponding to the log file.

[0069] This application also provides a specific use case for automatic repair of log binding IDs based on partially repairable codes. The following details the implementation in conjunction with this scenario. In the operation and maintenance management platform of a distributed system, development and testing personnel need to quickly locate fault logs through log pre-retrieval. When a user enters the log binding ID in the client for querying, the random code portion of the log binding ID may be incomplete or contain missing characters due to input errors, character loss, or network transmission anomalies, thus causing the integrity verification of the pre-retrieval system to fail. To address this issue, the system automatically triggers a log binding ID repair process based on partially repairable codes. The specific steps include: the system extracts the random code that failed the integrity check from the input log binding ID, i.e., the second random code; according to the generation rule of this random code, the system dynamically obtains the corresponding verification matrix, i.e., the second target matrix, to ensure that the repair operation conforms to the encoding constraints; the second random code is transposed into a column vector form to obtain the third random code, and a system of linear equations is established to satisfy "the product of the third random code and the second target matrix is ​​zero"; by solving this system of equations, the value of the missing character is calculated, and one or more possible repair schemes are obtained; the solved values ​​are matched one by one with the missing characters to complete the repair of the second random code; the integrity check of the repaired log binding ID is performed again, and after the check passes, the system accurately locates the corresponding node in the improved Merkle tree based on the repaired log binding ID; finally, the corresponding log data is extracted from the database and returned, supporting developers and testers to quickly locate faults. This embodiment greatly improves the fault tolerance and query accuracy of the log pre-retrieval system through an automatic repair mechanism, avoids retrieval failures caused by incomplete or incorrect log binding ID input, improves system stability and user experience, and is particularly suitable for large-scale, high-concurrency distributed log management environments.

[0070] This application also provides a specific use case for intelligent repair and integrity verification of log binding IDs based on locally repairable codes. The following is an application scenario: In the data center of a large internet company, system logs record detailed information about system operation, including transaction requests, responses, exceptions, and server status. To effectively manage and quickly retrieve these logs, a blockchain-based log pre-retrieval system is adopted, where the design of the log binding ID follows the principle of locally repairable codes. When technicians encounter an unknown system error in the test environment, due to negligence when manually recording the log binding ID, a few characters are missing from the random code portion of the ID, resulting in an incomplete input log binding ID that cannot be located by the log pre-retrieval system. In this case, an integrity verification can be performed by inputting the incomplete log binding ID, checking the length of the random code portion and detecting whether there are missing characters. Due to the length not meeting the requirements and the presence of missing characters, the integrity verification fails. The system recognizes the failure of the integrity verification and immediately initiates a repair mechanism. It first extracts the incomplete second random code portion. According to the storage rules, it retrieves the corresponding second target matrix, which has been pre-generated based on the principle of locally repairable codes and is used to repair and verify the random code portion. By multiplying a matrix by the transpose of a random code, the system can identify the possible values ​​of missing characters, calculate the possible missing characters, generate a repaired log binding ID, and perform another integrity check. If at least one of the repaired log binding IDs passes the integrity check, the system then performs a log pre-search on that ID. Based on an improved Merkle tree structure, it quickly locates the log file containing the specific timestamp, extracts key information, rapidly pinpoints the problem, and performs repair. This locally repairable code repair mechanism enables rapid retrieval, avoiding the tedious process of manual correction by developers and significantly improving the efficiency of problem localization.

[0071] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0072] This application also provides a log file pre-retrieval device. It should be noted that the log file pre-retrieval device of this application can be used to execute the log file pre-retrieval method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0073] The following describes the pre-retrieval device for log files provided in the embodiments of this application.

[0074] Figure 5 This is a structural block diagram of a log file pre-retrieval device according to an embodiment of this application. For example... Figure 5 As shown, the device includes:

[0075] The first acquisition unit 10 is used to receive the log binding ID input by the client and perform integrity verification on the log binding ID, wherein the log binding ID includes the log timestamp.

[0076] The integrity check includes length verification and verification of the correctness of the random code portion. If the length of the random code portion does not meet the preset length, or if a character is missing, the integrity check will fail.

[0077] The first retrieval unit 20 is used to traverse the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed the integrity check. The leaf node of the improved Merkle tree includes the hash value, timestamp and log binding ID corresponding to the log file. The timestamp of each node of the improved Merkle tree includes the timestamps of all its child nodes.

[0078] Specifically, based on the input log timestamp, the root node timestamps of all Merkle trees are traversed to find the root node that meets the condition, that is, the Merkle tree whose root node timestamp is greater than or equal to the input timestamp.

[0079] The determining unit 30 is used to determine the corresponding Merkle tree as the target Merkle tree when the first timestamp contains the log timestamp.

[0080] If the root node timestamp is the same as the input log timestamp, or if the range of the root node timestamp can cover the input timestamp, then the corresponding Merkle tree is marked as the target Merkle tree mentioned above, and becomes the object of the next retrieval.

[0081] The second retrieval unit 40 is used to traverse the target Merkle tree until the leaf node whose corresponding second timestamp is the same as the log timestamp is determined, and the target leaf node is obtained.

[0082] The leaf nodes of the Merkle tree contain the hash value, timestamp, and log binding ID of the log file. The system starts from the root node of the target Merkle tree and traverses down the tree level until it finds a leaf node that matches the input log timestamp. During traversal, the left subtree is checked first; if the timestamp range of the left subtree does not include the input timestamp, the system moves to the right subtree, and so on, until the target leaf node is found.

[0083] The second acquisition unit 50 is used to retrieve corresponding data from the database based on the target leaf node to obtain the target data.

[0084] In the above embodiments, firstly, the first acquisition unit receives the log binding ID input by the client and performs integrity verification on the log binding ID, which includes a log timestamp. Next, the first retrieval unit traverses the first timestamps of the root nodes of each improved Merkle tree based on the log timestamp of the log binding ID that passed integrity verification. Then, the determining unit determines the corresponding Merkle tree as the target Merkle tree if the first timestamp contains the log timestamp. Next, the second retrieval unit traverses the target Merkle tree until the leaf node with the same second timestamp as the log timestamp is determined, thus obtaining the target leaf node. Finally, the second acquisition unit retrieves the corresponding data from the database based on the target leaf node, thus obtaining the target data. This application enables the system to perform large-scale data retrieval operations by comparing the root node, internal nodes, and leaf nodes of the improved Merkle tree layer by layer using timestamps. This avoids traversing data blocks with mismatched timestamps. Compared to the existing technology that uses log-bound IDs to perform global traversal of the Merkle tree, this application reduces the amount of data traversed during the retrieval process and solves the problem of low retrieval efficiency of Merkle tree-based retrieval methods as the amount of data in the system increases.

[0085] In one alternative, the first acquisition unit includes: a first extraction module, used to extract the random code from the log binding ID to obtain a candidate random code;

[0086] The second extraction module is used to extract the encoding length of the above-mentioned candidate random codes to obtain the target length. If the target length is less than the preset length, it is determined that the integrity verification of the above-mentioned log binding ID fails. If the target length is equal to the preset length, it is determined that the integrity verification of the above-mentioned log binding ID passes.

[0087] The first iteration module iterates through each character of the candidate random code. If any of the characters are missing, the integrity check of the log binding ID is determined to be unsuccessful. If no missing characters are found, the integrity check of the log binding ID is determined to be successful.

[0088] In the above embodiments, a candidate random code is obtained by extracting a partially repairable code from the log binding ID input by the user. This allows the verification process to be based on the encoding structure of the candidate random code, ensuring that the retrieval request relies not only on a simple identifier but also on a random code with error correction and self-verification functions, thus improving the reliability of the input data. By extracting the encoding length of the candidate random code and comparing it with a preset length threshold, the integrity of the input log binding ID can be quickly determined. If the length is insufficient, the verification fails, avoiding resource consumption caused by invalid or erroneous input and improving system response speed and computational efficiency. By iterating through each character of the candidate random code, it checks for missing values. If missing characters are found, the log binding ID integrity verification fails, ensuring that only complete and correct log binding IDs can enter the subsequent retrieval process, effectively preventing query errors or inaccurate results due to missing data.

[0089] Specifically, missing values ​​include blank inputs, illegal characters, or uncertain symbols.

[0090] In another alternative embodiment, the first retrieval unit includes: a third extraction module, used to extract the random code from the log binding ID that has passed the integrity check, to obtain a first random code; a verification module, used to verify the first random code according to a first target matrix, wherein the first target matrix is ​​a verification matrix determined based on the generated first random code when the first random code is generated; and a second traversal module, used to traverse the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID when the first random code passes the verification.

[0091] In the above embodiments, a first random code is first extracted from the log binding ID that has passed integrity verification. Then, the first random code is verified using a first target matrix corresponding to the generation of this first random code, i.e., a verification matrix for locally repairable codes. This ensures that the input random code strictly conforms to the encoding rules, effectively identifying potential encoding errors or illegal tampering, and improving the security and reliability of the log binding ID. The first target matrix is ​​a verification matrix dynamically generated based on the encoding structure determined when the first random code is generated. The dynamically generated verification matrix more accurately reflects the encoding characteristics of the random code, improves error detection capabilities, ensures the authenticity and integrity of the random code in the log binding ID, and prevents illegal data from passing through the pre-retrieval process. Under the premise that the random code verification passes, an improved Merkle tree root node traversal based on log timestamps is performed. Only after the first random code passes verification is the timestamp information in the log binding ID used to traverse the first timestamp of each root node of the improved Merkle tree. This achieves rapid location of the block where the target log is located, reduces invalid or erroneous traversals, and improves retrieval accuracy and efficiency. By combining dynamic verification of random codes with timestamp traversal, attacks that maliciously forge or tamper with log binding IDs are effectively resisted, ensuring the security of the retrieval system. At the same time, the system remains stable and reliable under high concurrency and large data volume scenarios, enhancing the robustness of the overall log pre-retrieval platform.

[0092] In some exemplary embodiments, the above-mentioned verification module includes: a first transpose submodule, used to perform a transpose operation on the first random code to obtain a fourth random code; and a calculation submodule, used to calculate the product of the fourth random code and the first target matrix, and determine that the verification of the first random code fails if the product is not 0.

[0093] In the above embodiments, the first random code is transposed into a column vector to obtain the fourth random code, which is then multiplied with the first target matrix. This multiplication reflects whether the random code satisfies the linear constraint of the locally repairable code. A zero product vector means the random code fully conforms to the linear relationship specified by the check matrix. If the product is non-zero, it indicates that the random code violates the linear constraint, contains errors or incomplete information, and the verification fails. This scheme can accurately detect any illegal modifications, incorrect encodings, or data corruption in the random code, ensuring the mathematical integrity and encoding validity of the random code, and improving the security verification strength of the log binding ID. It also reduces manual intervention and repeated verification steps, increases verification speed, and further improves retrieval efficiency.

[0094] In some other exemplary embodiments, the second extraction module includes: an extraction submodule, configured to extract the random code of the log binding ID that failed the integrity check, and obtain a second random code; an acquisition submodule, configured to acquire a second target matrix, wherein the second target matrix is ​​a verification matrix determined based on the generated second random code when the second random code is generated; and a repair submodule, configured to perform a repair operation on the second random code based on the second target matrix.

[0095] In the above embodiments, when the log binding ID fails the integrity check, the problematic random code portion of the binding ID is first precisely extracted to obtain a second random code, achieving accurate location of the abnormal data. The second target matrix is ​​the verification matrix corresponding to the generation of the second random code, which is a matrix dynamically generated based on the encoding rules of the random code. By obtaining the second target matrix, it is ensured that the repair process is strictly executed according to the original encoding constraints, making the repair operation more accurate, avoiding repair failures or errors caused by matrix mismatch, and improving the repair accuracy. By using the second target matrix to perform the repair operation on the second random code, self-correction of the input log binding ID is achieved, enabling accurate recovery of valid information and maintaining normal system operation even when faced with partially damaged or incomplete log binding IDs. This avoids interruption of the entire retrieval process or invalidation of results due to minor defects in the input data, significantly improving the robustness and fault tolerance of the system. Moreover, the repair process is completed automatically without manual intervention, saving human resources and further improving the overall retrieval efficiency.

[0096] According to some alternative embodiments of this application, the above-mentioned repair submodule includes: a second transpose submodule, used to perform a transpose operation on the second random code to obtain a third random code; a solution submodule, used to solve for the value of the character with missing value in the third random code under the condition that the multiplication of the third random code with the second target matrix is ​​0, to obtain at least one target value; and an association submodule, used to associate the target value with the corresponding character to complete the repair of the second random code.

[0097] In the above embodiments, firstly, the input second random code is transposed to obtain a third random code. By defining a system of linear equations and solving this system, the specific values ​​of all missing characters in the random code are determined, thereby completing the missing or erroneous parts of the second random code. This achieves rapid and accurate repair of the random code, avoiding manual intervention and improving repair efficiency and quality. By associating the target value obtained from the solution with the corresponding missing characters in the second random code one by one, replacing the originally missing or unknown parts, a complete random code that conforms to the encoding rules is finally generated. This effectively solves the problem of retrieval failure caused by missing or corrupted data in log binding ID input, enabling the system to automatically recover valid query identifiers, improving the system's tolerance to abnormal input and stable operation capability, reducing retrieval interruptions caused by data errors, and improving processing efficiency and user experience.

[0098] In some other exemplary solutions of this application, the second retrieval unit includes a third traversal module, which, in the case that there is no leaf node in the target Merkle tree that is the same as the corresponding second timestamp and the log timestamp, traverses each of the improved Merkle trees according to the log binding ID until the leaf node corresponding to the log binding ID is the same as the input log binding ID is determined, and obtains the target leaf node.

[0099] In the above embodiments, a secondary retrieval mechanism based on log binding IDs is provided to address the anomaly where a leaf node with an exact timestamp match cannot be found in the target Merkle tree. This ensures that even if log timestamps are abnormal, erroneous, or lost, the target log can still be accurately located using the log binding ID, thus improving the system's fault tolerance. Using the complete input log binding ID as the retrieval keyword, all improved Merkle trees are traversed to accurately locate the corresponding log data in any block, reducing the risk of retrieval failures caused by timestamp errors, log delays, or anomalies. By completely matching the input log binding ID with the log binding IDs in the leaf nodes, the uniqueness and accuracy of the retrieval results are ensured, preventing incorrect matching and false detections, and guaranteeing the complete acquisition of critical log data.

[0100] The aforementioned log file pre-retrieval device includes a processor and a memory. The first acquisition unit, the first retrieval unit, the determining unit, the second retrieval unit, and the second acquisition unit are all stored as program units in the memory. The processor executes these program units stored in the memory to implement their respective functions. All of the aforementioned modules reside in the same processor; alternatively, the modules may be located in different processors in any combination.

[0101] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and communication efficiency can be improved by adjusting kernel parameters.

[0102] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0103] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the pre-retrieval method for the log file.

[0104] Specifically, the pre-retrieval methods for log files include:

[0105] Step S101: Receive the log binding ID input by the client and perform integrity verification on the log binding ID, which includes the log timestamp.

[0106] Step S102: Traverse the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed the integrity verification. The leaf node of the improved Merkle tree includes the hash value, timestamp and log binding ID corresponding to the log file. The timestamp of each node of the improved Merkle tree includes the timestamps of all its child nodes.

[0107] Step S103: If the first timestamp includes the log timestamp, the corresponding Merkle tree is determined as the target Merkle tree.

[0108] Step S104: Traverse the target Merkle tree until the leaf node whose corresponding second timestamp is the same as the log timestamp is determined, and obtain the target leaf node;

[0109] Step S105: Retrieve the corresponding data from the database based on the target leaf node to obtain the target data.

[0110] In one embodiment of this application, the log binding ID further includes an integrity check based on a random code with partial repairability, comprising: extracting the random code from the log binding ID to obtain a candidate random code; extracting the encoding length of the candidate random code to obtain a target length; determining that the log binding ID integrity check fails if the target length is less than a preset length, and determining that the log binding ID integrity check passes if the target length is equal to the preset length; traversing each character of the candidate random code, determining that the log binding ID integrity check fails if there is a missing character, and determining that the log binding ID integrity check passes if there is no missing character.

[0111] In one embodiment of this application, traversing the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed integrity verification includes: extracting the random code from the log binding ID that has passed integrity verification to obtain a first random code; verifying the first random code according to a first target matrix, wherein the first target matrix is ​​a verification matrix determined based on the generated first random code when generating the first random code; and if the first random code passes verification, traversing the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID.

[0112] In one embodiment of this application, the verification of the first random code based on the first target matrix includes: transposing the first random code to obtain a fourth random code; calculating the product of the fourth random code and the first target matrix; and determining that the verification of the first random code fails if the product is not zero.

[0113] In one embodiment of this application, after determining that the integrity verification of the log binding ID fails, the method further includes: extracting the random code of the log binding ID that failed the integrity verification to obtain a second random code; obtaining a second target matrix, wherein the second target matrix is ​​a verification matrix determined based on the generated second random code when the second random code is generated; and performing a repair operation on the second random code based on the second target matrix.

[0114] In one embodiment of this application, the repair operation of the second random code based on the second target matrix includes: transposing the second random code to obtain a third random code; solving for the value of the character with missing value in the third random code based on the condition that the multiplication of the third random code and the second target matrix is ​​0, to obtain at least one target value; associating the target value with the corresponding character to complete the repair of the second random code.

[0115] In one embodiment of this application, after traversing the target Merkle tree, the method further includes: if there is no leaf node in the target Merkle tree that is the same as the corresponding second timestamp and the log timestamp, traversing each of the improved Merkle trees according to the log binding ID until a leaf node corresponding to the log binding ID that is the same as the input log binding ID is determined, thereby obtaining the target leaf node.

[0116] This invention provides a system for running a program, wherein the program executes the pre-retrieval method for the log files during runtime.

[0117] Specifically, the pre-retrieval methods for log files include:

[0118] Step S101: Receive the log binding ID input by the client and perform integrity verification on the log binding ID, which includes the log timestamp.

[0119] Step S102: Traverse the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed the integrity verification. The leaf node of the improved Merkle tree includes the hash value, timestamp and log binding ID corresponding to the log file. The timestamp of each node of the improved Merkle tree includes the timestamps of all its child nodes.

[0120] Step S103: If the first timestamp includes the log timestamp, the corresponding Merkle tree is determined as the target Merkle tree.

[0121] Step S104: Traverse the target Merkle tree until the leaf node whose corresponding second timestamp is the same as the log timestamp is determined, and obtain the target leaf node;

[0122] Step S105: Retrieve the corresponding data from the database based on the target leaf node to obtain the target data.

[0123] In one embodiment of this application, the log binding ID further includes an integrity check based on a random code with partial repairability, comprising: extracting the random code from the log binding ID to obtain a candidate random code; extracting the encoding length of the candidate random code to obtain a target length; determining that the log binding ID integrity check fails if the target length is less than a preset length, and determining that the log binding ID integrity check passes if the target length is equal to the preset length; traversing each character of the candidate random code, determining that the log binding ID integrity check fails if there is a missing character, and determining that the log binding ID integrity check passes if there is no missing character.

[0124] In one embodiment of this application, traversing the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed integrity verification includes: extracting the random code from the log binding ID that has passed integrity verification to obtain a first random code; verifying the first random code according to a first target matrix, wherein the first target matrix is ​​a verification matrix determined based on the generated first random code when generating the first random code; and if the first random code passes verification, traversing the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID.

[0125] In one embodiment of this application, the verification of the first random code based on the first target matrix includes: transposing the first random code to obtain a fourth random code; calculating the product of the fourth random code and the first target matrix; and determining that the verification of the first random code fails if the product is not zero.

[0126] In one embodiment of this application, after determining that the integrity verification of the log binding ID fails, the method further includes: extracting the random code of the log binding ID that failed the integrity verification to obtain a second random code; obtaining a second target matrix, wherein the second target matrix is ​​a verification matrix determined based on the generated second random code when the second random code is generated; and performing a repair operation on the second random code based on the second target matrix.

[0127] In one embodiment of this application, the repair operation of the second random code based on the second target matrix includes: transposing the second random code to obtain a third random code; solving for the value of the character with missing value in the third random code based on the condition that the multiplication of the third random code and the second target matrix is ​​0, to obtain at least one target value; associating the target value with the corresponding character to complete the repair of the second random code.

[0128] In one embodiment of this application, after traversing the target Merkle tree, the method further includes: if there is no leaf node in the target Merkle tree that is the same as the corresponding second timestamp and the log timestamp, traversing each of the improved Merkle trees according to the log binding ID until a leaf node corresponding to the log binding ID that is the same as the input log binding ID is determined, thereby obtaining the target leaf node.

[0129] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0136] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0137] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0139] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0140] 1) The pre-retrieval method for log files in this application first receives a log binding ID input by the client and performs integrity verification on the log binding ID, which includes a log timestamp; then, based on the log timestamp of the log binding ID that has passed integrity verification, it traverses the first timestamp of the root node of each improved Merkle tree; then, if the first timestamp contains the log timestamp, the corresponding Merkle tree is determined as the target Merkle tree; next, the target Merkle tree is traversed until the leaf node with the same second timestamp as the log timestamp is determined, thus obtaining the target leaf node; finally, the corresponding data is retrieved from the database based on the target leaf node to obtain the target data. This application enables the system to perform large-scale data retrieval operations by comparing the root node, internal nodes, and leaf nodes of the improved Merkle tree layer by layer using timestamps. This avoids traversing data blocks with mismatched timestamps. Compared to the existing technology that uses log-bound IDs to perform global traversal of the Merkle tree, this application reduces the amount of data traversed during the retrieval process and solves the problem of low retrieval efficiency of Merkle tree-based retrieval methods as the amount of data in the system increases.

[0141] 2) The log file pre-retrieval device of this application first receives the log binding ID input by the client through the first acquisition unit and performs integrity verification on the log binding ID, which includes a log timestamp; then, the first retrieval unit traverses the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed the integrity verification; then, the determination unit determines the corresponding Merkle tree as the target Merkle tree if the first timestamp contains the log timestamp; then, the second retrieval unit traverses the target Merkle tree until the leaf node with the same second timestamp as the log timestamp is determined, and the target leaf node is obtained; finally, the second acquisition unit retrieves the corresponding data from the database according to the target leaf node to obtain the target data. This application enables the system to perform large-scale data retrieval operations by comparing the root node, internal nodes, and leaf nodes of the improved Merkle tree layer by layer using timestamps. This avoids traversing data blocks with mismatched timestamps. Compared to the existing technology that uses log-bound IDs to perform global traversal of the Merkle tree, this application reduces the amount of data traversed during the retrieval process and solves the problem of low retrieval efficiency of Merkle tree-based retrieval methods as the amount of data in the system increases.

[0142] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for pre-retrieval of log files, characterized in that, include: Receive the log binding ID input by the client and perform integrity verification on the log binding ID, which includes the log timestamp; The first timestamp of the root node of each improved Merkle tree is traversed according to the log timestamp of the log binding ID that has passed the integrity check. The leaf nodes of the improved Merkle tree include the hash value, timestamp and log binding ID corresponding to the log file. The timestamp of each node of the improved Merkle tree includes the timestamps of all its child nodes. If the first timestamp includes the log timestamp, the corresponding Merkle tree is determined as the target Merkle tree; Traverse the target Merkle tree until a leaf node whose corresponding second timestamp is the same as the log timestamp is determined, and the target leaf node is obtained; Based on the target leaf node, the corresponding data is retrieved from the database to obtain the target data.

2. The method according to claim 1, characterized in that, The log binding ID also includes an integrity check based on a random code for locally repairable functions, including: Extract the random code from the log binding ID to obtain the candidate random code; Extract the encoding length of the candidate random code to obtain the target length. If the target length is less than the preset length, it is determined that the log binding ID integrity verification fails. If the target length is equal to the preset length, it is determined that the log binding ID integrity verification passes. Iterate through each character of the candidate random code. If a character is missing a value, the integrity check of the log binding ID is determined to be unsuccessful. If no character is missing a value, the integrity check of the log binding ID is determined to be successful.

3. The method according to claim 2, characterized in that, Based on the log timestamp of the log binding ID that has passed integrity verification, traverse the first timestamp of the root node of each improved Merkle tree, including: Extract the random code from the log binding ID that has passed the integrity verification to obtain the first random code; The first random code is verified according to the first target matrix, wherein the first target matrix is ​​a verification matrix determined based on the generated first random code when the first random code is generated; If the first random code passes the verification, the first timestamp of the root node of each improved Merkle tree is traversed according to the log timestamp of the log binding ID.

4. The method according to claim 3, characterized in that, The first random code is verified based on the first target matrix, including: The first random code is transposed to obtain the fourth random code; Calculate the product of the fourth random code and the first target matrix. If the product is not zero, determine that the verification of the first random code has failed.

5. The method according to claim 2, characterized in that, After determining that the log binding ID integrity verification fails, the method further includes: Extract the random code from the log binding ID that failed the integrity check to obtain a second random code; Obtain the second target matrix, which is the verification matrix determined based on the generated second random code when generating the second random code; The second random code is repaired based on the second target matrix.

6. The method according to claim 5, characterized in that, The repair operation on the second random code based on the second target matrix includes: The third random code is obtained by transposing the second random code. Using the condition that the multiplication of the third random code and the second target matrix is ​​0, the values ​​of the characters in the third random code that have missing values ​​are solved to obtain at least one target value; The target value is associated with the corresponding character to complete the repair of the second random code.

7. The method according to claim 1, characterized in that, After traversing the target Merkle tree, the method further includes: If there is no leaf node in the target Merkle tree that is the same as the corresponding second timestamp and the log timestamp, the improved Merkle tree is traversed according to the log binding ID until the leaf node corresponding to the log binding ID is the same as the input log binding ID is determined, and the target leaf node is obtained.

8. A pre-retrieval device for log files, characterized in that, The device includes: The first acquisition unit is used to receive the log binding ID input by the client and perform integrity verification on the log binding ID, wherein the log binding ID includes a log timestamp; The first retrieval unit is used to traverse the first timestamp of the root node of each improved Merkle tree according to the log timestamp of the log binding ID that has passed the integrity check. The leaf node of the improved Merkle tree includes the hash value, timestamp and log binding ID corresponding to the log file. The timestamp of each node of the improved Merkle tree includes the timestamps of all its child nodes. The determining unit is configured to determine the corresponding Merkle tree as the target Merkle tree when the first timestamp includes the log timestamp; The second retrieval unit is used to traverse the target Merkle tree until the leaf node whose corresponding second timestamp is the same as the log timestamp is determined, and the target leaf node is obtained. The second acquisition unit is used to retrieve corresponding data from the database based on the target leaf node to obtain the target data.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 7.

10. A data storage system, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 7.

Images