Log Processing Method, Apparatus, Electronic Device, and Readable Storage Medium

Abstract

The present invention provides a log processing method, apparatus, electronic device, and readable storage medium. The log processing method includes obtaining a first log message corresponding to a first time period; saving the first log message to a distributed file system to obtain a first message fingerprint corresponding to the first log message; generating a second log message for a second time period, the second time period including N of the first time periods, N being an integer greater than 1, the second log message including the first message fingerprints of the N first time periods that make up the second time period; and saving the second log message to the distributed file system to obtain a second message fingerprint corresponding to the second log message. Embodiments of the present invention help to improve the organization and convenience of log management and storage.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of communication technologies, and in particular, to a log processing method, apparatus, electronic device, and readable storage medium. Background Art

[0002] With the development of communication technologies, the network has become an indispensable part of people's lives, and a large number of log records are generated every day by related network systems, network devices, network traffic, etc. Most of the related technical solutions adopt database or packaged compression storage solutions, which are inconvenient for managing log data. Summary of the Invention

[0003] Embodiments of the present invention provide a log processing method, apparatus, electronic device, and readable storage medium to solve the problem that it is inconvenient to manage log data in related technologies.

[0004] To solve the above problems, the present invention is implemented as follows:

[0005] In a first aspect, embodiments of the present invention provide a log processing method, the method including:

[0006] Obtain a first log packet corresponding to a first time period;

[0007] Save the first log packet to a distributed file system to obtain a first packet fingerprint corresponding to the first log packet, where the first packet fingerprint is unique, and the first log packet and the first packet fingerprint are in one-to-one correspondence;

[0008] Generate a second log packet for a second time period, the second time period including N first time periods, N being an integer greater than 1, the second log packet including the first packet fingerprints of the N first time periods that make up the second time period;

[0009] Save the second log packet to the distributed file system to obtain a second packet fingerprint corresponding to the second log packet, the second packet fingerprint being unique, and the second log packet and the second packet fingerprint being in one-to-one correspondence.

[0010] Optionally, when the first time period is the smallest time period, the obtaining a first log packet corresponding to the first time period includes:

[0011] Obtain the log data for the first time period;

[0012] Save the log data to a distributed file system to obtain a first log fingerprint corresponding to the log data, the first log fingerprint being unique, and the first log fingerprint and the log data being in one-to-one correspondence;

[0013] Generate a first log message including the first log fingerprint.

[0014] Optionally, after saving the second log message to the distributed file system, the method further includes:

[0015] Receive an access request for target log data in the log data;

[0016] Determine, according to the access request, a second target message fingerprint corresponding to the target log data in the second message fingerprint;

[0017] Search in the distributed file system for a second target log message corresponding to the second target message fingerprint according to the second target message fingerprint, where the second target log message is one of the second log messages;

[0018] Search for a first target message fingerprint corresponding to the target log data among the first message fingerprints included in the second target log message;

[0019] Search in the distributed file system for a first target log message corresponding to the first target message fingerprint according to the first target message fingerprint, where the first target log message is one of the first log messages;

[0020] In the case where the first time period is the minimum time period, obtain a first target log fingerprint according to the first target log message, where the first target log fingerprint is the log fingerprint corresponding to the target log data in the first log fingerprint;

[0021] Search for the target log data in the distributed file system according to the first target log fingerprint.

[0022] Optionally, generating the second log message for the second time period includes:

[0023] Establish a first fingerprint list, where the first fingerprint list includes first message fingerprints corresponding to N first time periods included in the second time period;

[0024] Use the first fingerprint list as the message body of the second log message to generate a second log message.

[0025] Optionally, after generating the second log message for the second time period, the method further includes:

[0026] Clear the first message fingerprints corresponding to N first time periods included in the second time period.

[0027] In a second aspect, an embodiment of the present invention provides a log processing device, including:

[0028] An acquisition module, configured to acquire a first log message corresponding to a first time period;

[0029] A first storage module, configured to store the first log message in a distributed file system, and obtain a first message fingerprint corresponding to the first log message, where the first message fingerprint is unique, and the first log message and the first message fingerprint are in one-to-one correspondence;

[0030] A generation module, configured to generate a second log message for a second time period, where the second time period includes N first time periods, N is an integer greater than 1, and the second log message includes first message fingerprints of the N first time periods that make up the second time period;

[0031] A second storage module, configured to store the second log message in the distributed file system, and obtain a second message fingerprint corresponding to the second log message, where the second message fingerprint is unique, and the second log message and the second message fingerprint are in one-to-one correspondence.

[0032] Optionally, when the first time period is the minimum time period, the acquisition module includes:

[0033] A log data acquisition sub-module, configured to acquire log data for the first time period;

[0034] A storage sub-module, configured to store the log data in a distributed file system, and obtain a first log fingerprint corresponding to the log data, where the first log fingerprint is unique, and the first log fingerprint and the log data are in one-to-one correspondence;

[0035] A log message generation sub-module, configured to generate a first log message including the first log fingerprint.

[0036] Optionally, it further includes: an access request receiving module, configured to receive an access request for target log data in the log data;

[0037] A first search module, configured to determine, according to the access request, a second target message fingerprint corresponding to the target log data among the second message fingerprints;

[0038] A second search module, configured to search, in the distributed file system according to the second target message fingerprint, for a second target log message corresponding to the second target message fingerprint, where the second target log message is one of the second log messages;

[0039] A third search module, configured to search for a first target message fingerprint corresponding to the target log data from the first message fingerprints included in the second target log message;

[0040] A fourth search module, configured to search for a first target log message in the distributed file system according to the first target message fingerprint, where the first target log message is one of the first log messages;

[0041] A fifth search module, configured to, when the first time period is the minimum time period, obtain a first target log fingerprint according to the first target log message, where the first target log fingerprint is a log fingerprint corresponding to the target log data among the first log fingerprints;

[0042] A sixth search module, configured to search for the target log data in the distributed file system according to the first target log fingerprint.

[0043] Optionally, the generation module includes:

[0044] A fingerprint list establishment sub-module, configured to establish a first fingerprint list, where the first fingerprint list includes first message fingerprints corresponding to N first time periods included in the second time period;

[0045] A generation sub-module, configured to use the first fingerprint list as the message body of the second log message to generate a second log message.

[0046] Optionally, it further includes: a clearing module, configured to clear the first message fingerprints corresponding to N first time periods included in the second time period.

[0047] In a third aspect, an embodiment of the present invention further provides a communication device, including: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps in the method described in the foregoing first aspect.

[0048] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium, configured to store a program, where the program, when executed by a processor, implements the steps in the method described in the foregoing first aspect.

[0049] In the embodiment of the present invention, a second log message is generated according to the first message fingerprint including the first log message, so that the message fingerprints of different hierarchical time periods can be saved in sequence. When accessing, queries can be made level by level according to the corresponding time periods, which helps to improve the organization and convenience of log management and storage. Description of the Drawings

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0051] Figure 1 is a schematic flowchart of the log processing method provided by an embodiment of the present invention;

[0052] Figure 2 is a schematic structural diagram of the log processing method device provided by an embodiment of the present invention;

[0053] Figure 3 is a schematic structural diagram of the electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0055] The terms "first", "second", etc. in the embodiments of the present invention are used to distinguish similar objects and do not necessarily describe a specific order or sequence. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices. In addition, the use of "and / or" in this application means at least one of the connected objects. For example, A and / or B and / or C means including A alone, B alone, C alone, as well as the cases where A and B both exist, B and C both exist, A and C both exist, and A, B, and C all exist.

[0056] The following describes the log processing method provided by the embodiments of the present invention.

[0057] See Figure 1 , Figure 1 which is a schematic flowchart of the log processing method provided by an embodiment of the present invention.

[0058] In one embodiment, the log processing method includes:

[0059] Step 101: Obtain a first log message corresponding to a first time period.

[0060] In this embodiment, the first time period can be set as needed. For example, it can be a time period of different durations such as 15 minutes, one day, or one week. In this embodiment, the log message corresponding to the first time period is denoted as the first log message, and this first log message is used to obtain the log data of the first time period.

[0061] Step 102: Save the first log message to the distributed file system to obtain the first message fingerprint corresponding to the first log message.

[0062] The distributed file system in this embodiment can be IPFS (InterPlanetary File System). IPFS is a protocol and distributed network file system that uses content-addressable and peer-to-peer transmission methods for file transmission. It has characteristics such as permanent validity of resources, non-tampering, and very small redundancy. IPFS is based on a peer-to-peer system, with high data distribution efficiency, and has the characteristics of anti-single point of failure and anti-DDoS, which helps to save server resources and maintenance costs and improve the availability of files.

[0063] IPFS adopts a peer-to-peer transmission strategy, which can divide a file into file blocks and generate corresponding hash values as fingerprint information to mark the file or file block, so as to uniquely identify the file or file block. Through this file fingerprint information, the corresponding file or file block can be addressed, searched for, and downloaded in the IPFS network.

[0064] During use, a unique hash value can be generated for a file in a certain node in the IPFS system, and the file can be published to the IPFS system, and other nodes in the system can download the file according to the hash value.

[0065] In one embodiment, the first log message can be saved to IPFS and a corresponding hash value can be generated as the first message fingerprint of the first log message. This first message fingerprint is unique, and the first log message and the first message fingerprint are in one-to-one correspondence.

[0066] Step 103: Generate a second log message for the second time period.

[0067] The second time period includes N first time periods, where N is an integer greater than 1. For example, if the first time period is defined as 15 minutes, then the second time period can be defined as one day; or if the first time period is defined as one day, then the second time period can be defined as one month.

[0068] In this way, the second time period is composed of N first time periods. Therefore, the log data corresponding to this second time period is actually jointly constituted by the log data of these N first time periods.

[0069] The second log message generated in this embodiment includes the first message fingerprints of the N first time periods that make up the second time period. In this way, since the first message fingerprint and the first log message are in one-to-one correspondence, according to the first message fingerprints included in the second log message, the corresponding first log messages can be queried accordingly. Therefore, the corresponding first log messages can be determined based on the generated second log message, and thus the corresponding log data can be determined.

[0070] Step 104: Save the second log message to the distributed file system to obtain the second message fingerprint corresponding to the second log message.

[0071] Next, save the second log message to the distributed file system, for example, it can be saved to IPFS, to obtain the second message fingerprint corresponding to the second log message. Similarly, the second message fingerprint is unique, and the second log message and the second message fingerprint are in one-to-one correspondence.

[0072] In the embodiment of the present invention, the second log message is generated according to the first message fingerprint including the first log message, so that the message fingerprints of different hierarchical time periods can be saved in sequence. When accessing, it can be queried level by level according to the corresponding time period, which helps to improve the organization and convenience of log management and storage.

[0073] It should be noted that the first time period and the second time period are not fixed. For example, in a log processing process, every 15 minutes is used as a first time period, and one day is used as a second time period to generate the log message of each day.

[0074] After generating the log messages of multiple days, in order to further organize the corresponding log data, each day is used as a first time period, and each week is used as a second time period, and the log message of each week can be generated. By analogy, the log messages of different time periods such as one month, one quarter, one year, and other different lengths can be generated.

[0075] In the technical solution of this embodiment, the time period can include multiple levels. Exemplarily, in one embodiment, the time period includes five levels: 15 minutes, day, week, month, and year. Obviously, the specific length and levels of the time period can be adjusted accordingly according to needs.

[0076] In the process of performing the above steps 101 to 104, 15 minutes can be first used as the above first time period, and a day can be used as the above second time period to obtain the log messages and message fingerprints for each day.

[0077] Next, using a day as the above first time period and a week as the above second time period, the log messages and message fingerprints for each week can be obtained. By analogy, the log messages and message fingerprints for different hierarchical time periods can be obtained.

[0078] In some of the embodiments, when the first time period is the smallest time period, for example, when it is the above 15 minutes, there are no lower-level log messages for this first time period. Therefore, corresponding log messages need to be generated when the first time period is the smallest time period.

[0079] In one of the embodiments, when the first time period is the smallest time period, the above step 101 includes:

[0080] Obtain the log data for the first time period;

[0081] Save the log data to a distributed file system to obtain a first log fingerprint corresponding to the log data;

[0082] Generate a first log message including the first log fingerprint.

[0083] In this embodiment, saving the log data of the smallest time period to a distributed file system can be, for example, the IPFS system. In this way, the first log fingerprint corresponding to the log data of the smallest time period can be obtained. The first log fingerprint saved in the distributed file system is also unique, and the first log fingerprint and the log data are in one-to-one correspondence.

[0084] Next, generate a first log message according to the first log fingerprint. Specifically, the first log fingerprint can be used as the body of the message, and the header information can be added accordingly to generate the first log message.

[0085] In some of the embodiments, the header information of the log message, such as the header information of the first log message or the second log message, includes a serial number, a message level, a timestamp, a time period, a geographical identifier, a company identifier, a device identifier, and a custom field.

[0086] The serial number is used to mark the order of generation, starting from 1 until the message is merged into a higher-level log data message and the counting restarts.

[0087] Message levels correspond to time periods of different levels. Exemplarily, when the time periods include five levels of 15 minutes, day, week, month, and year, their levels are defined as T0, T1, T2, T3, and T4 in sequence. The log message of 15 minutes corresponds to level T0, and the value of this message level is 0. Obviously, in actual implementation, it can be represented in other ways. For example, it can also be defined as three levels of TA, TB, and TC, and ABC can be used as the corresponding levels.

[0088] The timestamp represents the time when the first log message is generated.

[0089] The time period identifier indicates that all logs within the unit time period (such as within 15 minutes) recorded by this first log message. Combining with the timestamp, the specific time period covered by the logs of this Ticket can be calculated.

[0090] The geographical location identifier is used to identify the geographical location information where the first log message is generated. For example, when counting the log data in a certain office building, it can be used to identify different offices; when counting the log data of the branches of a certain company in different locations, it can be accurate to different regions, etc. At this time, this geographical location identifier can be represented by a geographical location code.

[0091] The company code identifier is used to identify the company information where the first log message is generated, indicating that the logs recorded at level T0 are generated and recorded by this company. It can be represented by the company credit code, or a company list can be established and a unique label can be given to each company.

[0092] The log collection device identifier is used to clarify the device number that generates the first log message, facilitating the traceability of the device information where the data is generated, and can be represented by the device hardware identifier.

[0093] The custom attribute is to facilitate customization requirements and reserve a part of the space to record more specific attribute information.

[0094] During implementation, one or more of the above items can be selected as the header information, and more information can be added to the custom attribute to generate the log message.

[0095] In some of these embodiments, generating the second log message of the second time period includes:

[0096] Establishing a first fingerprint list, where the first fingerprint list includes the first message fingerprints corresponding to N first time periods included in the second time period;

[0097] Using the first fingerprint list as the message body of the second log message to generate the second log message.

[0098] The second time period is composed of N first time periods. Therefore, the second time period is necessarily not the minimum time period. When generating the second log message of the second time period, the list of first message fingerprints corresponding to the N first time periods included in the second time period is used as the message body, and the header information is added accordingly to form the second log message. Among them, the header information can refer to the generation method of the header information of the log message in the above embodiment, so that the second log message including the first fingerprint information can be generated.

[0099] In some embodiments, after step 104, the method further includes:

[0100] Receiving an access request for target log data in the log data;

[0101] According to the access request, determining, among the second message fingerprints, a second target message fingerprint corresponding to the target log data;

[0102] According to the second target message fingerprint, searching in the distributed file system for a second target log message corresponding to the second target message fingerprint, where the second target log message is one of the second log messages;

[0103] Searching, among the first message fingerprints included in the second target log message, for a first target message fingerprint corresponding to the target log data;

[0104] According to the first target message fingerprint, searching in the distributed file system for a first target log message, where the first target log message is one of the first log messages;

[0105] In the case where the first time period is the minimum time period, obtaining a first target log fingerprint according to the first target log message, where the first target log fingerprint is the log fingerprint corresponding to the target log data among the first log fingerprints;

[0106] Searching for the target log data in the distributed file system according to the first target log fingerprint.

[0107] This embodiment can be understood as the access process for log data.

[0108] When it is necessary to access the log data of a certain specific time period, the access request must carry the corresponding time information, and the corresponding time period can be directly determined or determined after conversion according to this time period.

[0109] For example, if you want to access the log data for the first day of the Xth week in the Nth month of year Y, the corresponding time period is directly carried in the range request. If you want to access the log data for the 3rd day of the Nth month of year Y, and this day is Wednesday, then the 3rd day of the Nth month of year Y can be converted to the third day of the 1st week in the Nth month of year Y, so as to obtain its corresponding time period.

[0110] Next, find the corresponding second message fingerprint according to the time period. For example, it can be found that the message fingerprint of year Y is used as the second target message fingerprint from multiple second message fingerprints such as the message fingerprints of year Y - 1, year Y, and year Y + 1.

[0111] Further, find the corresponding second target log message from the distributed system according to the second target message fingerprint, that is, the log message of the Yth year. The body of the log message of the Yth year includes the message fingerprints of January to December, which are the above-mentioned first message fingerprints. Find the message fingerprint of the Nth month from them, which is the first message fingerprint.

[0112] Next, use the first target message fingerprint as the new second message fingerprint, and iterate the above process until the message at the specified time is found.

[0113] And so on, until the smallest time period is found. At this time, the body of the first target log is the log fingerprint of the corresponding log data. The log data can be found according to this log fingerprint. In this way, the access to the log data for a specific time period is realized.

[0114] In some embodiments, after generating the second log message of the second time period, the method further includes:

[0115] Delete the first message fingerprints corresponding to the N first time periods included in the second time period.

[0116] After generating the second log message, the second log message includes the first message fingerprints corresponding to the corresponding N first time periods. Therefore, if you need to find these first log fingerprints, you can directly find them from the second log message. At this time, delete the original first message data. In this way, data loss will not occur, and at the same time, storage space can be saved, as well as the maintenance cost for the first log fingerprint can be saved.

[0117] In the technical solution of this embodiment, a hierarchical management solution is adopted by combining log data with a distributed file system such as IPFS. Taking the log data packet as the maintenance object, a hierarchical log data packet is formed. When querying logs, only the high-level log data packet covering the query time range needs to be found, and then all the log data of each unit time period within the entire time range covered by it can be found hierarchically in IPFS, reducing the response pressure of the blockchain. In addition, the technical solution of this embodiment maintains and establishes multiple blockchains according to the time dimension, which can reduce the frequency of uploading to the chain.

[0118] Based on the IPFS distributed storage technology, this solution can also reduce the number of query requests to the central server during the log query process, reducing the pressure on the central server. In addition, the control instructions such as query requests and the data instructions such as data download operations are isolated from each other. Relying on the powerful and efficient distribution and storage capabilities of IPFS, while improving the storage elasticity of log data, it also improves the efficiency of obtaining log data, and further reduces the pressure on the central server to handle both query instructions and docking download services.

[0119] In terms of log anti-tampering, it is proposed to rely on blockchain technology to achieve log anti-tampering by uploading the fingerprint generated when the log is uploaded to the chain. To further reduce the pressure of uploading to the blockchain, a method of uploading fingerprint data to the chain is proposed to relieve the pressure of uploading to the blockchain. To reduce the frequency of uploading to the chain, the maintained blockchains are divided into four types: daily chain, weekly chain, monthly chain, and annual chain according to the time dimension, and corresponding maintenance mechanisms are established to reduce the frequency of uploading to the chain.

[0120] For easy understanding, the example is described as follows:

[0121] Taking the maintenance of the log data of a certain group company as an example for illustration.

[0122] During the maintenance process, the minimum log collection unit period is defined as 15 minutes, and the log collection period also includes days, weeks, months, and years. The log maintenance period is divided into five levels: 0, 1, 2, 3, and 4. Among them, level 0 corresponds to the minimum log collection unit period, that is, 15 minutes, level 1 corresponds to days, level 2 corresponds to weeks, and so on.

[0123] After the log collection device completes the log data collection every 15 minutes, it uploads the log data d0 of these 15 minutes to the IPFS network, and the fingerprint corresponding to this log data is called f0. The device uses this f0 as the text data of T0 and supplements the header information of T0 according to the specified format of the T0 packet to form the T0 packet.

[0124] Exemplarily, the log data generated by Company A every 15 minutes from 00:00 to 23:59 in a day are d0(A1), d0(A2)... d0(An) respectively. Among them, d0(A1) refers to the log data of Company A in the time period from 00:00 to 00:15 on that day, d0(A2) refers to the log data of Company A in the time period from 00:15 to 00:30 on that day... and so on.

[0125] Upload the data of d0(A1), d0(A2)... d0(An) to the IPFS network to obtain the corresponding fingerprints f0(A1), f0(A2)... f0(An). Among them, f0(A1) refers to the fingerprint of the log data of Company A in the time period from 00:00 to 00:15 on that day, f0(A2) refers to the fingerprint of the log data of Company A in the time period from 00:15 to 00:30 on that day... and so on.

[0126] Next, take f0(A1), f0(A2)... f0(An) as the body texts of the log data messages in each time period of Company A on that day respectively, and supplement the corresponding header attribute information to form T0(A1), T0(A2)... T0(An). Among them, T0(A1) refers to the log data message of Company A in the time period from 00:00 to 00:15 on that day, T0(A2) refers to the log data message of Company A in the time period from 00:15 to 00:30 on that day, and similarly for T0(An).

[0127] Upload T0(A1), T0(A2)……T0(An) to the IPFS network to obtain the corresponding fingerprints t0(A1), t0(A2)... t0(An). Among them, t0(A1) refers to the fingerprint of the log data message of Company A in the time period from 00:00 to 00:15 on that day, t0(A2) refers to the fingerprint of the log data message of Company A in the time period from 00:15 to 00:30 on that day... and so on.

[0128] Form a fingerprint list of t0(A1), t0(A2)……t0(An) as the body of the log data message T1 of Company A on that day, and supplement the corresponding attribute information as the header information, denoted as T1(A1). Similarly, the log data message generated on the second day is denoted as T1(A2)……and so on.

[0129] In this way, after the end of the first day, T1(A1) is obtained, and after the end of the second day, T1(A2) is obtained... and so on. After the obtained T1(A1), T1(A2)……T1(A7) are uploaded to the chain, it can be understood as the "daily chain" in the blockchain data.

[0130] After the end of the first week, T1(A1), T1(A2) …… T1(A7) can be obtained. At this time, T1(A1), T1(A2) …… T1(A7) are uploaded to the IPFS network to obtain the corresponding fingerprint data t1(A1), t1(A2)... t1(A7).

[0131] Next, t1(A1), t1(A2)... t1(A7) are formed into a fingerprint list as the body of the log data message T2 of Company A this week, and the corresponding attribute information is supplemented as the header information to obtain the log message of this week, denoted as T2(A1). At this time, the corresponding T1(A1), T1(A2)... T1(A7) content can be obtained from the IPFS network through the fingerprint information in the body of T2(A1). At this time, only the T2(A1) message needs to be left, and the T1(A1), T1(A2)... T1(A7) messages can be deleted. After the obtained T2(A1), T2(A2) …… T2(A7) are chained, it can be understood as the "weekly chain" in the blockchain. It can be understood that at this time, only the "weekly chain" needs to be maintained, and the corresponding "daily chain" no longer needs to be maintained.

[0132] By analogy, when the time dimension reaches one month, a series of T2 data can be integrated into T3 data and the original series of T2 data can be deleted at the same time, that is, the corresponding "monthly chain" is maintained, and the corresponding "weekly chain" no longer needs to be maintained; when the time dimension reaches one year, a series of T3 data can be integrated into T4 data and the original series of T3 data can be deleted at the same time, that is, the corresponding "yearly chain" is maintained, and the corresponding "monthly chain" no longer needs to be maintained.

[0133] Furthermore, in a similar way, the relevant log data of Company B in the group company can be obtained, such as T2(B1), T3(B1) and the log data T2(C1) of Company C, and so on. By analogy, the summary, storage and collation of the log data of each company in the group company in each time dimension can be realized.

[0134] When relevant log data needs to be queried, for example, when a log auditor needs to query relevant log data, the log auditor queries the blockchain, queries the attribute information such as the specified time, location, unit, etc., obtains the index address of the data message, obtains the corresponding message in the blockchain, and indexes the content of the specified message layer by layer to find the corresponding unit fingerprint data T0.

[0135] Exemplarily, if there is a full-year fingerprint data message T4 of a certain Company X in 2019 in the yearly chain, if you want to index the corresponding log data of this company from 10:15 to 10:30 on April 15, 2019, the following search method can be adopted:

[0136] 1) The log auditor obtains the annual fingerprint data message T4 of Company X in 2019 from the "Year Chain" blockchain and finally obtains T4.

[0137] 2) The log auditor finds the fingerprint data message T3 of Company X in April 2019 in the IPFS network through the list of log data fingerprints of each month of Company X in 2019 in the body of T4.

[0138] 3) The log auditor finds the fingerprint data message T2 of the third week of April 2019 of Company X in the IPFS network through the list of log data fingerprints of each week of Company X in April 2019 in the body of T3.

[0139] 4) The log auditor finds the fingerprint data message T1 of the 15th day, which is the first day of the second week of April 2019 of Company X in the IPFS network through the list of log data fingerprints of each day of the second week of April 2019 of Company X in the body of T2.

[0140] 5) The log auditor finds the fingerprint data message T0 of the 42nd 15 - minute time period from 10:15 to 10:30 on April 15, 2019 of Company X in the IPFS network through the list of log data fingerprints of each 15 - minute time unit on April 15, 2019 of Company X in the body of T1.

[0141] 6) The log auditor finds the corresponding log data of Company X from 10:15 to 10:30 on April 15, 2019 in the IPFS network according to the body data of T0.

[0142] Referring to the above method, the storage and query of log data can be realized.

[0143] See Figure 2 , Figure 2 is one of the structural diagrams of the log processing device provided by the embodiment of the present invention. As Figure 2 shown, the log processing device 200 includes:

[0144] An obtaining module 201, configured to obtain a first log message corresponding to a first time period;

[0145] A first saving module 202, configured to save the first log message to a distributed file system to obtain a first message fingerprint corresponding to the first log message, where the first message fingerprint is unique, and the first log message and the first message fingerprint are in one - to - one correspondence;

[0146] A generation module 203, configured to generate a second log message for a second time period, where the second time period includes N first time periods, N is an integer greater than 1, and the second log message includes first message fingerprints of the N first time periods that constitute the second time period;

[0147] A second storage module 204, configured to store the second log message into the distributed file system, to obtain a second message fingerprint corresponding to the second log message, where the second message fingerprint is unique, and the second log message and the second message fingerprint are in one-to-one correspondence.

[0148] In some embodiments, when the first time period is the minimum time period, the acquisition module 201 includes:

[0149] A log data acquisition sub-module, configured to acquire log data for the first time period;

[0150] A storage sub-module, configured to store the log data into the distributed file system, to obtain a first log fingerprint corresponding to the log data, where the first log fingerprint is unique, and the first log fingerprint and the log data are in one-to-one correspondence;

[0151] A log message generation sub-module, configured to generate a first log message including the first log fingerprint.

[0152] In some embodiments, it further includes: an access request receiving module, configured to receive an access request for target log data in the log data;

[0153] A first search module, configured to determine, according to the access request, a second target message fingerprint corresponding to the target log data among the second message fingerprints;

[0154] A second search module, configured to search, according to the second target message fingerprint, in the distributed file system for a second target log message corresponding to the second target message fingerprint, where the second target log message is one of the second log messages;

[0155] A third search module, configured to search, among the first message fingerprints included in the second target log message, for a first target message fingerprint corresponding to the target log data;

[0156] A fourth search module, configured to search, according to the first target message fingerprint, in the distributed file system for a first target log message, where the first target log message is one of the first log messages;

[0157] A fifth search module, configured to, when the first time period is the minimum time period, obtain a first target log fingerprint according to the first target log message, where the first target log fingerprint is a log fingerprint corresponding to the target log data in the first log fingerprints;

[0158] A sixth search module, configured to search for the target log data in the distributed file system according to the first target log fingerprint.

[0159] In some embodiments, the generating module 203 includes:

[0160] A fingerprint list establishing sub-module, configured to establish a first fingerprint list, where the first fingerprint list includes first message fingerprints corresponding to N first time periods included in the second time period;

[0161] A generating sub-module, configured to generate a second log message by using the first fingerprint list as the message body of the second log message.

[0162] In some embodiments, it further includes: a clearing module, configured to clear the first message fingerprints corresponding to N first time periods included in the second time period.

[0163] The log processing device 200 can implement each process in the method embodiments of the present invention Figure 1 and achieve the same beneficial effects. To avoid repetition, details are not described herein again.

[0164] Embodiments of the present invention further provide an electronic device. Please refer to Figure 3 , the electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and executable on the processor 301.

[0165] When the program 3021 is executed by the processor 901, it can implement Figure 1 any step in the corresponding method embodiment and achieve the same beneficial effects. Details are not described herein again.

[0166] Those of ordinary skill in the art can understand that all or part of the steps for implementing the method in the above embodiments can be completed by hardware related to program instructions, and the program can be stored in a readable medium. Embodiments of the present invention further provide a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement any step in the above Figure 3 corresponding method embodiment, and can achieve the same technical effects. To avoid repetition, details are not described herein again.

[0167] The storage medium described above, such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0168] The above is the preferred implementation mode of the embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle described in the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A log processing method, characterized in that, The method includes: Obtaining a first log message corresponding to a first time period; Saving the first log message to a distributed file system, and obtaining a first message fingerprint corresponding to the first log message, where the first message fingerprint is unique, and the first log message and the first message fingerprint are in one-to-one correspondence; Generating a second log message for a second time period, the second time period including N of the first time periods, N being an integer greater than 1, and the second log message including the first message fingerprints of the N first time periods that make up the second time period; Saving the second log message to the distributed file system, and obtaining a second message fingerprint corresponding to the second log message, the second message fingerprint being unique, and the second log message and the second message fingerprint being in one-to-one correspondence.

2. The method according to claim 1, characterized in that, When the first time period is the smallest time period, the obtaining a first log message corresponding to the first time period includes: Obtaining log data for the first time period; Saving the log data to a distributed file system, and obtaining a first log fingerprint corresponding to the log data, the first log fingerprint being unique, and the first log fingerprint and the log data being in one-to-one correspondence; Generating a first log message including the first log fingerprint.

3. The method according to claim 2, wherein After saving the second log message to the distributed file system, the method further includes: Receiving an access request for target log data in the log data; Determining, according to the access request, a second target message fingerprint in the second message fingerprint that corresponds to the target log data; Searching, according to the second target message fingerprint, in the distributed file system for a second target log message corresponding to the second target message fingerprint, the second target log message being one of the second log messages; Searching, among the first message fingerprints included in the second target log message, for a first target message fingerprint corresponding to the target log data; Searching, according to the first target message fingerprint, in the distributed file system for a first target log message, the first target log message being one of the first log messages; When the first time period is the smallest time period, obtaining a first target log fingerprint according to the first target log message, the first target log fingerprint being the log fingerprint in the first log fingerprint that corresponds to the target log data; Searching for the target log data in the distributed file system according to the first target log fingerprint.

4. The method according to claim 1, wherein The generating a second log message for a second time period includes: Establishing a first fingerprint list, the first fingerprint list including the first message fingerprints corresponding to the N first time periods included in the second time period; Using the first fingerprint list as the message body of the second log message to generate a second log message.

5. The method according to claim 1 or 4, characterized in that After generating the second log message for the second time period, the method further includes: Clearing the first message fingerprints corresponding to the N first time periods included in the second time period.

6. A log processing device, characterized in that, including: An acquisition module, configured to acquire a first log message corresponding to a first time period; A first storage module, configured to store the first log message into a distributed file system, and obtain a first message fingerprint corresponding to the first log message, where the first message fingerprint is unique, and the first log message and the first message fingerprint are in one-to-one correspondence; A generation module, configured to generate a second log message for a second time period, where the second time period includes N first time periods, N is an integer greater than 1, and the second log message includes the first message fingerprints of the N first time periods that constitute the second time period; A second storage module, configured to store the second log message into the distributed file system, and obtain a second message fingerprint corresponding to the second log message, where the second message fingerprint is unique, and the second log message and the second message fingerprint are in one-to-one correspondence.

7. The device according to claim 6, characterized in that, When the first time period is the minimum time period, the acquisition module includes: A log data acquisition sub-module, configured to acquire log data of the first time period; A storage sub-module, configured to store the log data into a distributed file system, and obtain a first log fingerprint corresponding to the log data, where the first log fingerprint is unique, and the first log fingerprint and the log data are in one-to-one correspondence; A log message generation sub-module, configured to generate a first log message including the first log fingerprint.

8. The device according to claim 7, characterized in that, It further includes: An access request receiving module, configured to receive an access request for target log data in the log data; A first search module, configured to determine, according to the access request, a second target message fingerprint corresponding to the target log data among the second message fingerprints; A second search module, configured to search, according to the second target message fingerprint, in the distributed file system for a second target log message corresponding to the second target message fingerprint, where the second target log message is one of the second log messages; A third search module, configured to search, among the first message fingerprints included in the second target log message, for a first target message fingerprint corresponding to the target log data; A fourth search module, configured to search, according to the first target message fingerprint, in the distributed file system for a first target log message, where the first target log message is one of the first log messages; A fifth search module, configured to, when the first time period is the minimum time period, obtain a first target log fingerprint according to the first target log message, where the first target log fingerprint is the log fingerprint corresponding to the target log data among the first log fingerprints; A sixth search module, configured to search, according to the first target log fingerprint, in the distributed file system for the target log data.

9. The device according to claim 6, characterized in that, The generation module includes: A fingerprint list establishment sub-module, configured to establish a first fingerprint list, where the first fingerprint list includes the first message fingerprints corresponding to the N first time periods included in the second time period; A generating sub-module, configured to use the first fingerprint list as the message body of the second log message to generate a second log message.

10. The device according to claim 6 or 9, characterized in that, It further includes: A clearing module, configured to clear the first message fingerprints corresponding to the N first time periods included in the second time period.

11. An electronic device, comprising: A transceiver, a memory, a processor, and a program stored on the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the log processing method according to any one of claims 1 to 5.

12. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the log processing method according to any one of claims 1 to 5.

Images