InterPlanetary File System IPFS, Data Storage Method, Device and Communication Node
By introducing cache areas and index information on-chain storage in IPFS, the problem of log data being tampered with before writing in the prior art is solved, real-time tamper-proof and secure storage of log data is realized, and real-time storage of streaming data is supported.
Patent Information
- Application Number
- CN202110001508.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-04
- Publication Date
- 2025-07-22
- Estimated Expiration
- 2041-01-04
AI Technical Summary
The existing IPFS-based data storage solution cannot support real-time writing, and there is a risk that log data will be maliciously tampered with before writing, resulting in inaccurate log data and losing the meaning of secure storage.
The interstellar file system IPFS is adopted, including IPFS storage area, cache area and acquisition interface. The log data is received in real time through the acquisition interface and cached in the cache area. It is not stored until the preset data amount is reached and then moved to the IPFS storage area. The index information of the log data is generated and stored on the chain.
Real-time tamper-proof and secure storage of log data is realized, ensuring that the log data is immutable from generation to IPFS storage process, avoiding malicious tampering of log data before writing, and supporting the real-time storage function of streaming data.
Smart Images

Figure CN114721580B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communication technologies, and in particular, to an InterPlanetary File System (IPFS), a data storage method, an apparatus, and a communication node. Background Art
[0002] Log auditing plays a crucial role in enterprise security construction, enabling post-event accountability and traceability, and contributing to enterprise security construction. In traditional log auditing systems, there is a risk that logs can be tampered with, deleted, or forged by privileged account personnel, rendering log auditing meaningless. Therefore, technical means are needed to prevent log tampering. For this, a storage solution based on IPFS (InterPlanetary File System) is currently provided.
[0003] However, the existing IPFS-based storage solution cannot support real-time writing of file content. In application scenarios where logs are generated in real time, there is a risk that they may be tampered with by malicious users (such as super administrators) before being written to IPFS, resulting in inaccurate stored log data and rendering the use of IPFS for secure storage meaningless.
[0004] As can be seen from the above, the existing data storage solutions still have problems in ensuring security. Summary of the Invention
[0005] The purpose of the present invention is to provide an InterPlanetary File System (IPFS), a data storage method, an apparatus, and a communication node to solve the problem that the existing data storage solutions cannot ensure security.
[0006] To solve the above technical problems, an embodiment of the present invention provides an InterPlanetary File System (IPFS), including:
[0007] An IPFS storage area, a buffer area capable of communicating with the IPFS storage area, and a collection interface capable of communicating with the buffer area;
[0008] Wherein, the collection interface is used to receive real-time log data generated in real time; the buffer area is used to cache the log data received in real time by the collection interface without storing it; and the IPFS storage area is used to store the log data cached by the buffer area.
[0009] Optionally, the IPFS storage area is connected to the buffer area through a first connection line; and / or,
[0010] The buffer area is connected to the collection interface through a second connection line.
[0011] Optionally, the collection interface performs data transmission through a preset remote access protocol;
[0012] Among them, the preset remote access protocol includes at least one of the Network File System (NFS) protocol, the File Transfer Protocol (FTP), and the Hypertext Transfer Protocol (HTTP).
[0013] An embodiment of the present invention further provides a data storage method, which is applied to a first communication node including the above-mentioned InterPlanetary File System (IPFS). The method includes:
[0014] Receiving, in real time, log data sent by a second communication node through a collection interface in the IPFS, caching the log data in a buffer area in the IPFS, and not storing it;
[0015] When the log data cached in the buffer area reaches a preset data volume, moving the cached log data to an IPFS storage area in the IPFS for storage.
[0016] Optionally, the moving the cached log data to an IPFS storage area in the IPFS for storage includes:
[0017] Transmitting the cached log data to the IPFS storage area for storage and clearing the buffer area.
[0018] Optionally, after moving the cached log data to an IPFS storage area in the IPFS for storage, it further includes:
[0019] Returning to execute the operation of receiving, in real time, log data sent by a second communication node through a collection interface in the IPFS, caching the log data in a buffer area in the IPFS, and not storing it.
[0020] Optionally, it further includes:
[0021] Generating index information of the log data moved from the buffer area to the IPFS storage area;
[0022] Storing the index information in a blockchain.
[0023] An embodiment of the present invention further provides a data storage method, which is applied to a second communication node. The method includes:
[0024] Obtaining log data generated in real time;
[0025] Sending the log data to a collection interface in the IPFS of a first communication node in real time.
[0026] An embodiment of the present invention further provides a data storage device, which is applied to a first communication node including the above-mentioned InterPlanetary File System (IPFS). The device includes:
[0027] The first processing module is used to receive the log data sent by the second communication node in real time through the acquisition interface in the IPFS, cache it in the buffer area in the IPFS, and not store it.
[0028] The second processing module is used to move the cached log data to the IPFS storage area in the IPFS for storage when the amount of log data cached in the buffer area reaches a preset data volume.
[0029] Optionally, the second processing module includes:
[0030] The first processing sub-module is used to transfer the cached log data to the IPFS storage area for storage and clear the buffer area.
[0031] Optionally, it further includes:
[0032] The first execution module is used to return and execute the operation of receiving the log data sent by the second communication node in real time through the acquisition interface in the IPFS, caching it in the buffer area in the IPFS, and not storing it after moving the cached log data to the IPFS storage area in the IPFS for storage.
[0033] Optionally, it further includes:
[0034] The first generation module is used to generate index information of the log data moved from the buffer area to the IPFS storage area;
[0035] The first storage module is used to store the index information in the blockchain.
[0036] An embodiment of the present invention further provides a data storage device, which is applied to the second communication node. The device includes:
[0037] The first acquisition module is used to acquire the log data generated in real time;
[0038] The first sending module is used to send the log data to the acquisition interface in the IPFS of the first communication node in real time.
[0039] An embodiment of the present invention further provides a communication node. The communication node is the first communication node including the above-mentioned InterPlanetary File System (IPFS). The communication node includes a processor and a transceiver;
[0040] The processor is used to receive the log data sent by the second communication node in real time through the acquisition interface in the IPFS by using the transceiver, cache it in the buffer area in the IPFS, and not store it;
[0041] When the log data cached in the buffer reaches a preset data volume, move the cached log data to the IPFS storage area in the IPFS for storage.
[0042] Optionally, the processor is specifically configured to:
[0043] Transmit the cached log data to the IPFS storage area for storage, and clear the buffer.
[0044] Optionally, the processor is further configured to:
[0045] After moving the cached log data to the IPFS storage area in the IPFS for storage, return to execute the operation of receiving in real time the log data sent by the second communication node through the acquisition interface in the IPFS, caching the log data in the buffer in the IPFS, and not storing it.
[0046] Optionally, the processor is further configured to:
[0047] Generate index information of the log data moved from the buffer to the IPFS storage area;
[0048] Store the index information in the blockchain.
[0049] An embodiment of the present invention further provides a communication node, which is a second communication node, and the communication node includes: a processor and a transceiver;
[0050] The processor is configured to obtain log data generated in real time;
[0051] Send the log data to the acquisition interface in the IPFS of the first communication node in real time through the transceiver.
[0052] An embodiment of the present invention further provides a communication node, including a memory, a processor, and a program stored on the memory and executable on the processor; when the processor executes the program, it implements the data storage method on the side of the first communication node as described above; or,
[0053] When the processor executes the program, it implements the data storage method on the side of the second communication node as described above.
[0054] An embodiment of the present invention further provides a readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements the steps in the data storage method on the side of the first communication node as described above; or,
[0055] When the program is executed by a processor, it implements the steps in the data storage method on the side of the second communication node as described above.
[0056] The beneficial effects of the above technical solutions of the present invention are as follows:
[0057] In the above solution, the IPFS is provided with an IPFS storage area, a buffer area capable of communicating with the IPFS storage area, and a collection interface capable of communicating with the buffer area. Among them, the collection interface is used to receive in real time the log data generated in real time; the buffer area is used to cache the log data received in real time by the collection interface and does not store it; the IPFS storage area is used to store the log data cached by the buffer area. It can support the direct real-time transmission of the generated log data through the collection interface to the buffer area of the IPFS, and use the buffer area to cache the log data collected in real time. When the buffer area reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, enabling the IPFS to realize the function of real-time storage of streaming data. In addition, since the log data is directly transmitted to the buffer area of the IPFS through the collection interface, and the data in the buffer area is difficult to be tampered with, the integrity and security of the log data from generation to IPFS storage are ensured, avoiding malicious tampering of the log data before being written into the IPFS, and realizing real-time anti-tampering of the log data. It well solves the problem that the data storage solution in the prior art cannot ensure security. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 It is a schematic diagram of the IPFS structure according to an embodiment of the present invention;
[0059] Figure 2 It is a schematic diagram of the data storage method process according to an embodiment of the present invention Figure 1 ;
[0060] Figure 3 It is a schematic diagram of the data storage method process according to an embodiment of the present invention Figure 2 ;
[0061] Figure 4 It is a schematic diagram of the implementation architecture of the data storage method according to an embodiment of the present invention;
[0062] Figure 5 It is a schematic diagram of the IPFS application according to an embodiment of the present invention;
[0063] Figure 6 It is a schematic diagram of the structure of the data storage device according to an embodiment of the present invention Figure 1 ;
[0064] Figure 7 It is a schematic diagram of the structure of the data storage device according to an embodiment of the present invention Figure 2 ;
[0065] Figure 8 It is a schematic diagram of the structure of the communication node according to an embodiment of the present invention Figure 1 ;
[0066] Figure 9Schematic diagram of the communication node structure according to an embodiment of the present invention Figure 2 。 Detailed implementation manners
[0067] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.
[0068] In view of the problem that the existing data storage scheme cannot ensure security in the prior art, the present invention provides an InterPlanetary File System (IPFS), as Figure 1 shown, including:
[0069] An IPFS storage area 1, a buffer area 2 capable of communicating with the IPFS storage area 1, and a collection interface 3 capable of communicating with the buffer area 2; wherein, the collection interface 3 is used to receive in real time the log data generated in real time; the buffer area 2 is used to cache the log data received in real time by the collection interface 3 and is not (written to disk) stored; the IPFS storage area 1 is used to (write to disk) store the log data cached by the buffer area 2.
[0070] The IPFS provided by the embodiment of the present invention is provided with an IPFS storage area, a buffer area capable of communicating with the IPFS storage area, and a collection interface capable of communicating with the buffer area; wherein, the collection interface is used to receive in real time the log data generated in real time; the buffer area is used to cache the log data received in real time by the collection interface and is not stored; the IPFS storage area is used to store the log data cached by the buffer area; it can support the generated log data to be directly transmitted in real time to the buffer area of the IPFS through the collection interface, and the buffer area is used to cache the log data collected in real time. When the buffer area reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, enabling the IPFS to achieve the function of real-time storage of streaming data; in addition, since the log data is directly transmitted to the buffer area of the IPFS through the collection interface, and the data in the buffer area is difficult to be tampered with, the integrity and security of the log data from generation to storage in the IPFS are ensured, avoiding malicious tampering of the log data before being written to the IPFS, and realizing real-time anti-tampering of the log data; it well solves the problem that the existing data storage scheme cannot ensure security in the prior art.
[0071] Wherein, the IPFS storage area and the buffer area can be connected by a first connection line; and / or, the buffer area and the collection interface can be connected by a second connection line.
[0072] In the embodiment of the present invention, the collection interface can perform data transmission through a preset remote access protocol; wherein, the preset remote access protocol includes at least one of the Network File System (NFS) protocol, the File Transfer Protocol (FTP), and the HyperText Transfer Protocol (HTTP).
[0073] An embodiment of the present invention further provides a data storage method, which is applied to a first communication node including the above-mentioned InterPlanetary File System (IPFS), as Figure 2 shown. The method includes:
[0074] Step 21: Receive the log data sent by the second communication node in real time through the acquisition interface in the IPFS, cache it in the buffer area in the IPFS, and do not store it;
[0075] Step 22: When the log data cached in the buffer area reaches a preset data volume, move the cached log data to the IPFS storage area in the IPFS for storage.
[0076] Among them, the preset data volume can be an integer multiple of 256 KB, or KB of any positive integer, which is not limited here. Step 21 may specifically include: receiving the log data sent by the second communication node in real time through the acquisition interface in the IPFS by using a preset remote access protocol; the preset remote access protocol includes at least one of the Network File System (NFS) protocol, the File Transfer Protocol (FTP), and the HyperText Transfer Protocol (HTTP).
[0077] The data storage method provided by the embodiment of the present invention receives the log data sent by the second communication node in real time through the acquisition interface in the IPFS, caches it in the buffer area in the IPFS, and does not store it; when the log data cached in the buffer area reaches a preset data volume, move the cached log data to the IPFS storage area in the IPFS for storage; it can be realized that the generated log data is directly transmitted to the buffer area of the IPFS in real time through the acquisition interface, and the buffer area is used to cache the log data collected in real time. When the buffer area reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, so that the IPFS realizes the function of real-time storage of streaming data; in addition, since the log data is directly transmitted to the buffer area of the IPFS through the acquisition interface, and the data in the buffer area is difficult to be tampered with, the integrity and security of the log data from generation to IPFS storage are ensured, the malicious tampering of the log data before writing into the IPFS is avoided, and the real-time anti-tampering of the log data is realized; it well solves the problem that the data storage scheme in the prior art cannot ensure security.
[0078] In the embodiment of the present invention, the moving the cached log data to the IPFS storage area in the IPFS for storage includes: transmitting the cached log data to the IPFS storage area for storage, and clearing the buffer area.
[0079] Further, after moving the cached log data to the IPFS storage area in the IPFS for storage, it further includes: returning to execute the operation of receiving in real time the log data sent by the second communication node through the collection interface in the IPFS, caching the log data in the buffer area in the IPFS, and not storing it.
[0080] It can also be understood that when the log volume in the buffer area reaches n KB (for example: 256 KB), the entire content of the cache block is then passed to the IPFS storage area for disk storage. At the same time, the buffer area is cleared to wait for receiving newly generated log data subsequently.
[0081] In the embodiment of the present invention, the data storage method further includes: generating index information of the log data moved from the buffer area to the IPFS storage area; storing the index information in the blockchain.
[0082] Specifically, storing the index information in the blockchain may include: signing the index information; storing the signed index information in the blockchain.
[0083] The embodiment of the present invention also provides a data storage method, which is applied to the second communication node, as Figure 3 shown, the method includes:
[0084] Step 31: Obtain the log data generated in real time;
[0085] Step 32: Send the log data to the collection interface in the IPFS of the first communication node in real time. Step 32 may specifically include: using a preset remote access protocol to send the log data to the collection interface in the IPFS of the first communication node in real time; the preset remote access protocol includes at least one of the Network File System NFS protocol, the File Transfer Protocol FTP, and the Hypertext Transfer Protocol HTTP.
[0086] The data storage method provided by the embodiment of the present invention obtains the log data generated in real time; and sends the log data to the collection interface in IPFS of the first communication node in real time; it can support the realization that the generated log data is directly transmitted to the buffer area of IPFS through the collection interface in real time, and the buffer area is used to cache the log data collected in real time. When the buffer area reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, enabling IPFS to realize the function of real-time storage of streaming data; in addition, since the log data is directly transmitted to the buffer area of IPFS through the collection interface, and the data in the buffer area is difficult to be tampered with, the integrity and security of the log data from generation to IPFS storage are ensured, avoiding malicious tampering of the log data before writing into IPFS, and realizing real-time anti-tampering of the log data; it well solves the problem that the data storage scheme in the prior art cannot ensure security.
[0087] The data storage method provided by the embodiment of the present invention will be further described below in combination with multiple parties such as the first communication node and the second communication node.
[0088] In view of the above technical problems, the embodiment of the present invention provides a data storage method, which can be specifically implemented by using the system architecture of a log anti-tampering system based on IPFS and blockchain as shown in Figure 4 The system includes four parts: a business system (corresponding to the second communication node above), IPFS (corresponding to the first communication node above), a blockchain system, and a log audit system. Among them, the business system generates log data in real time; IPFS is responsible for collecting and storing log data in real time. At the same time, fingerprint information (corresponding to the index information above) is generated, and then the fingerprint information is signed and uploaded to the blockchain; the blockchain system is responsible for verifying the signature and storing the fingerprint information; the log audit system is responsible for obtaining the fingerprint information from the blockchain system, then obtaining the log data from IPFS according to the fingerprint information, and performing normal audit of the log.
[0089] The operations of the entire system on the log data can be executed according to the following process:
[0090] 1. Real-time log collection
[0091] The user generates a log record by operating the business system. By using the IPFS provided by the embodiment of the present invention, the generated log record can be stored in IPFS in real time, without waiting to upload the complete log file to IPFS for storage. On the premise of secure transmission of the log file, the integrity of the data from generation to IPFS storage is ensured.
[0092] Specifically, the IPFS provided by the embodiment of the present invention is as shown in Figure 5As shown, IPFS can be configured to support the NFS (Network File System) protocol, enabling log data to be transmitted to IPFS through the log collection interface (corresponding to the above-mentioned collection interface); and a buffer is added in front of the IPFS storage area, so that the real-time collected log data is first cached in the buffer through the log collection interface. When the log volume in the buffer reaches n KB (corresponding to the above-mentioned preset data volume, for example: 256 KB), the entire buffer block content is then passed to the IPFS storage area for disk storage. At the same time, the buffer is cleared to wait for receiving subsequent newly generated log data. This solution realizes the real-time storage function of IPFS for streaming data.
[0093] Among them: NFS allows devices in the network to share resources through the TCP / IP (Internet communication protocol) network. In the application of NFS, the local NFS client application can transparently read and write files located on the remote NFS server as if accessing local files; it can also be understood that: NFS is a mechanism that mounts partitions (directories) on remote hosts to the local system via the network. Through the support of the network file system, users can operate on the shared partitions (directories) of remote hosts on the local system as if operating on local partitions.
[0094] It should be noted that in the embodiments of the present invention, "configuring IPFS to support NFS" can specifically be to configure the storage path of the log data collected by the collection interface as the path of the remote server, and the memory of this remote server is the above-mentioned buffer; subsequently, the log data cached (not yet stored on disk) in the buffer can be transmitted to the IPFS storage area for storage, that is, after the log is stored in the memory of the remote server (i.e., the buffer), IPFS then calls this area for disk storage. Among them, it involves adjusting IPFS to support the direct upload of in-memory data: modifying the specific instruction method of IPFS to enable IPFS to support the direct upload of in-memory data. The specific means can adopt existing methods and will not be elaborated here; in addition, in the embodiments of the present invention, the storage mechanism of the buffer includes that the data in the buffer is not stored on disk, and is directly pushed to IPFS after meeting the above size. The specific implementation of the storage mechanism can adopt existing methods and will not be elaborated here.
[0095] In the embodiments of the present invention, IPFS can also be configured to support remote access protocols such as FTP (File Transfer Protocol) and HTTP (Hypertext Transfer Protocol), enabling log data to be transmitted to IPFS through the log collection interface, which is not limited here.
[0096] Compared with the prior art of storing logs in existing IPFS, the storage process is as follows: The business system generates log data and directly stores it as a log file, and then the log file is transmitted to existing IPFS for disk storage. In this process, the directly stored log file may be tampered with by malicious users (because the log file generation process takes a long time, accessing the log file is relatively easy, and the operation of tampering with the log file has a low difficulty and a high possibility), resulting in the logs transmitted to IPFS having been tampered with.
[0097] In this solution, the generated logs are transmitted in real time to the buffer area of IPFS, and it is very difficult for malicious users to tamper with the data in the buffer area (because the data is stored in the buffer area for a short time and accessing the buffer area is difficult, and the operation of tampering with the buffer area has a very high difficulty and a very low possibility). Thus, the integrity of the logs from generation to storage in IPFS is ensured.
[0098] 2. Log Data Storage
[0099] IPFS stores the content of the cache block and simultaneously generates fingerprint information for the log file (corresponding to the cache block content). Then, the fingerprint information is signed and uploaded to the blockchain system.
[0100] Storing the log file in IPFS can achieve the integrity of the log file itself, ensuring that the log file cannot be modified or deleted. Further, to prevent malicious users from using tampered fingerprint information corresponding to the log to replace the real fingerprint information and mislead the log audit system to audit the tampered log file, in this solution, the fingerprint information is uploaded to the blockchain to ensure its integrity.
[0101] 3. Storing Fingerprint Information on the Blockchain
[0102] The blockchain system verifies the signature of the fingerprint information. If the verification passes, it is uploaded to the blockchain (one node in the blockchain can be used as the main node for verification, and then broadcast to inform each node, and each node then performs on-chain storage). Uploading the fingerprint information to the blockchain storage can ensure the integrity of the fingerprint information, and thus achieve the integrity of the log file.
[0103] 4. Log Audit
[0104] 4.1: When the log audit system conducts log audit, it first reads the fingerprint information from the blockchain; 4.2: Then it obtains data from IPFS for auditing; specifically: it reads the corresponding log file in IPFS according to the fingerprint information, and then performs normal log audit processing on the log file.
[0105] As can be seen from the above, the solution provided by the embodiment of the present invention involves:
[0106] 1. Provide an IPFS: Make it support the NFS protocol so that the generated log data can be directly transmitted to the IPFS, and set a buffer area in front of the IPFS storage area to cache the log data collected in real time. When the buffer area reaches the specified size, it is then directly transmitted to the IPFS storage area for disk storage; enabling the IPFS to achieve the function of real-time storage of streaming data (which can also be understood as real-time writing of streaming data into the IPFS).
[0107] 2. The log data is directly transmitted to the buffer area of the IPFS through the collection interface. Since it is very difficult to tamper with the data in the buffer area, the integrity of the log data from generation to IPFS storage is ensured. Subsequently, through operations such as storing the log data in the IPFS and fingerprinting on the blockchain, real-time anti-tampering of the log data is jointly achieved.
[0108] 3. Store a large number of log files (corresponding to the content of the cache blocks) themselves in the IPFS, and then store a small amount of fingerprint information corresponding to the log files in the blockchain, which not only realizes the immutability of the log data but also reduces the operating cost and storage cost of the blockchain system, and is feasible.
[0109] Generally speaking, this solution has the following advantages:
[0110] 1. This solution realizes the real-time storage function of streaming data in the IPFS: In this solution, by transforming the IPFS to make it support the NFS protocol and setting a buffer area to cache the log data collected in real time. When the buffer area reaches the specified size, the entire data in the buffer area is then transmitted to the IPFS for storage, realizing the function of real-time writing of streaming data into the IPFS.
[0111] 2. This solution ensures the real-time immutability of the log data: In this solution, by transforming the IPFS, the log data can be directly transmitted to the buffer area of the IPFS, and since it is very difficult and highly unlikely to tamper with the buffer area, the problem that the log data may be tampered with before being written into the IPFS is solved, realizing the real-time immutability of the log data.
[0112] 3. This solution has strong feasibility: This solution first stores the log data in the IPFS and then stores the generated fingerprint information in the blockchain, reducing the operating cost and storage cost of the blockchain, and has strong feasibility.
[0113] The embodiment of the present invention also provides a data storage device, which is applied to a first communication node including the above-mentioned InterPlanetary File System IPFS, as Figure 6 shown, the device includes:
[0114] A first processing module 61, configured to receive in real time the log data sent by a second communication node through the collection interface in the IPFS, and cache it in the buffer area in the IPFS without storing it;
[0115] A second processing module 62, configured to move the cached log data to an IPFS storage area in the IPFS for storage when the log data cached in the buffer reaches a preset data volume.
[0116] The data storage device provided by the embodiment of the present invention receives, through the acquisition interface in the IPFS, the log data sent by the second communication node in real time, caches it in the buffer in the IPFS, and does not store it; when the log data cached in the buffer reaches a preset data volume, the cached log data is moved to the IPFS storage area in the IPFS for storage; it can be realized that the generated log data is directly transmitted to the buffer of the IPFS in real time through the acquisition interface, and the buffer is used to cache the log data collected in real time. When the buffer reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, enabling the IPFS to realize the function of real-time storage of streaming data; in addition, since the log data is directly transmitted to the buffer of the IPFS through the acquisition interface, and the data in the buffer is difficult to be tampered with, the integrity and security of the log data from generation to IPFS storage are ensured, avoiding malicious tampering of the log data before it is written into the IPFS, and realizing real-time anti-tampering of the log data; it well solves the problem that the data storage scheme in the prior art cannot ensure security.
[0117] In the embodiment of the present invention, the second processing module includes: a first processing sub-module, configured to transmit the cached log data to the IPFS storage area for storage and clear the buffer.
[0118] Further, the data storage device further includes: a first execution module, configured to, after moving the cached log data to the IPFS storage area in the IPFS for storage, return to execute the operation of receiving, through the acquisition interface in the IPFS, the log data sent by the second communication node in real time, caching it in the buffer in the IPFS, and not storing it.
[0119] In the embodiment of the present invention, the data storage device further includes: a first generation module, configured to generate index information of the log data moved from the buffer to the IPFS storage area; a first storage module, configured to store the index information in a blockchain.
[0120] Among them, the implementation embodiments of the data storage method on the first communication node side are all applicable to the embodiment of this data storage device, and can also achieve the corresponding same technical effects.
[0121] The embodiment of the present invention further provides a data storage device, which is applied to a second communication node, as Figure 7As shown, the device includes:
[0122] A first acquisition module 71, configured to acquire real-time generated log data;
[0123] A first sending module 72, configured to send the log data to the acquisition interface in the IPFS of the first communication node in real time.
[0124] The data storage device provided by the embodiment of the present invention acquires real-time generated log data; sends the log data to the acquisition interface in the IPFS of the first communication node in real time; can support the realization that the generated log data is directly transmitted to the buffer area of the IPFS through the acquisition interface, and the buffer area is used to cache the real-time acquired log data. When the buffer area reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, enabling the IPFS to realize the function of real-time storage of streaming data; in addition, since the log data is directly transmitted to the buffer area of the IPFS through the acquisition interface, and the data in the buffer area is difficult to be tampered with, the integrity and security of the log data from generation to IPFS storage are ensured, avoiding malicious tampering of the log data before writing to the IPFS, and realizing real-time anti-tampering of the log data; it well solves the problem that the data storage scheme in the prior art cannot ensure security.
[0125] Among them, the implementation embodiments of the data storage method on the second communication node side are all applicable to the embodiments of this data storage device, and can also achieve the corresponding same technical effects.
[0126] The embodiment of the present invention also provides a communication node, and the communication node is a first communication node including the above-mentioned InterPlanetary File System IPFS, as Figure 8 shown, the communication node includes: a processor 81 and a transceiver 82;
[0127] The processor 81 is configured to use the transceiver 82 to receive the log data sent by the second communication node in real time through the acquisition interface in the IPFS, and cache it in the buffer area in the IPFS without storing it;
[0128] When the log data cached in the buffer area reaches the preset data volume, the cached log data is moved to the IPFS storage area in the IPFS for storage.
[0129] In the embodiment of the present invention, the communication node receives the log data sent by the second communication node in real time through the acquisition interface in the IPFS, caches it in the buffer area in the IPFS, and does not store it; when the log data cached in the buffer area reaches the preset data volume, the cached log data is moved to the IPFS storage area in the IPFS for storage; it can be realized that the generated log data is directly transmitted to the buffer area of the IPFS in real time through the acquisition interface, and the buffer area is used to cache the log data collected in real time. When the buffer area reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, enabling the IPFS to realize the function of real-time storage of streaming data; in addition, since the log data is directly transmitted to the buffer area of the IPFS through the acquisition interface, and the data in the buffer area is difficult to be tampered with, the integrity and security of the log data from generation to IPFS storage are ensured, avoiding malicious tampering of the log data before writing it into the IPFS, and realizing real-time anti-tampering of the log data; it well solves the problem that the data storage scheme in the prior art cannot ensure security.
[0130] In the embodiment of the present invention, the processor is specifically used for: transmitting the cached log data to the IPFS storage area for storage, and clearing the buffer area.
[0131] Further, the processor is also used for: after moving the cached log data to the IPFS storage area in the IPFS for storage, returning to execute the operation of receiving the log data sent by the second communication node in real time through the acquisition interface in the IPFS, caching it in the buffer area in the IPFS, and not storing it.
[0132] In the embodiment of the present invention, the processor is also used for: generating index information of the log data moved from the buffer area to the IPFS storage area; storing the index information in the blockchain.
[0133] Among them, the implementation embodiments of the data storage method on the first communication node side are all applicable to the embodiments of this communication node, and can also achieve the corresponding same technical effects.
[0134] The embodiment of the present invention also provides a communication node, and the communication node is the second communication node, as Figure 9 shown, the communication node includes: a processor 91 and a transceiver 92;
[0135] The processor 91 is used for obtaining the log data generated in real time;
[0136] Transmitting the log data to the acquisition interface in the IPFS of the first communication node in real time through the transceiver 92.
[0137] In the embodiment of the present invention, the communication node obtains the log data generated in real time, and sends the log data to the acquisition interface in the IPFS of the first communication node in real time. It can support the realization that the generated log data is directly transmitted to the buffer area of the IPFS through the acquisition interface, and the buffer area is used to cache the log data collected in real time. When the buffer area reaches the specified size, it is directly transmitted to the IPFS storage area for disk storage, enabling the IPFS to realize the function of real-time storage of streaming data. In addition, since the log data is directly transmitted to the buffer area of the IPFS through the acquisition interface, and the data in the buffer area is difficult to be tampered with, the integrity and security of the log data from generation to IPFS storage are ensured, avoiding malicious tampering of the log data before it is written into the IPFS, and realizing real-time anti-tampering of the log data. It well solves the problem that the data storage scheme in the prior art cannot ensure security.
[0138] Among them, the implementation embodiments of the data storage method on the second communication node side are applicable to the embodiments of this communication node, and can also achieve the corresponding same technical effects.
[0139] The embodiment of the present invention also provides a communication node, including a memory, a processor, and a program stored on the memory and executable on the processor. When the processor executes the program, it implements the data storage method on the first communication node side; or, when the processor executes the program, it implements the data storage method on the second communication node side.
[0140] Among them, the implementation embodiments of the data storage method on the first communication node side or the second communication node side are applicable to the embodiments of this communication node, and can also achieve the corresponding same technical effects.
[0141] The embodiment of the present invention also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, it implements the steps in the data storage method on the first communication node side; or, when the program is executed by a processor, it implements the steps in the data storage method on the second communication node side.
[0142] Among them, the implementation embodiments of the data storage method on the first communication node side or the second communication node side are applicable to the embodiments of this readable storage medium, and can also achieve the corresponding same technical effects.
[0143] It should be noted that many functional components described in this specification are referred to as modules / sub-modules in order to more particularly emphasize the independence of their implementation methods.
[0144] In the embodiments of the present invention, a module / sub-module may be implemented in software so as to be executed by various types of processors. For example, an identified executable code module may include one or more physical or logical blocks of computer instructions. For example, it may be constructed as an object, a procedure, or a function. Nevertheless, the executable code of the identified module need not be physically located together, but may include different instructions stored in different locations. When these instructions are logically combined together, they constitute the module and achieve the specified purpose of the module.
[0145] In fact, the executable code module may be a single instruction or many instructions, and may even be distributed over multiple different code segments, distributed among different programs, and distributed across multiple memory devices. Similarly, the operation data may be identified within the module, and may be implemented in any suitable form and organized within any suitable type of data structure. The operation data may be collected as a single data set, or may be distributed at different locations (including on different storage devices), and may at least partially exist only as electronic signals in the system or network.
[0146] When a module can be implemented in software, considering the level of existing hardware technology, for a module that can be implemented in software, without considering cost, those skilled in the art can build corresponding hardware circuits to implement the corresponding functions. The hardware circuits include conventional very large scale integration (VLSI) circuits or gate arrays, and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented using programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, etc.
[0147] The above are the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.
Claims
1. An InterPlanetary File System (IPFS), characterized in that, Including: An IPFS storage area, a buffer area capable of communicating with the IPFS storage area, and an acquisition interface capable of communicating with the buffer area; Wherein, the acquisition interface is used to receive in real time the log data generated in real time; the buffer area is used to cache the log data received in real time by the acquisition interface and does not store it; the IPFS storage area is used to store the log data cached by the buffer area; The IPFS is configured to support the Network File System (NFS) protocol; wherein, configuring the IPFS to support the NFS protocol includes: configuring the storage path of the log data collected by the acquisition interface as the path of a remote server, and the memory of the remote server serves as the buffer area.
2. The InterPlanetary File System IPFS according to claim 1, wherein The IPFS storage area is connected to the buffer area through a first connection line; and / or, The buffer area is connected to the acquisition interface through a second connection line.
3. The InterPlanetary File System IPFS according to claim 1, wherein The acquisition interface performs data transmission through a preset remote access protocol; Wherein, the preset remote access protocol includes at least one of the Network File System (NFS) protocol, the File Transfer Protocol (FTP), and the Hypertext Transfer Protocol (HTTP).
4. A data storage method, applied to a first communication node of the InterPlanetary File System (IPFS) described in any one of claims 1 to 3, characterized in that, The method includes: Receiving in real time, through the acquisition interface in the IPFS, the log data sent by a second communication node, caching it in the buffer area in the IPFS, and not storing it; When the log data cached in the buffer area reaches a preset data volume, moving the cached log data to the IPFS storage area in the IPFS for storage.
5. The data storage method according to claim 4, wherein After moving the cached log data to the IPFS storage area in the IPFS for storage, it further includes: Returning to execute the operation of receiving in real time, through the acquisition interface in the IPFS, the log data sent by a second communication node, caching it in the buffer area in the IPFS, and not storing it.
6. The data storage method according to claim 4, wherein It further includes: Generating index information of the log data moved from the buffer area to the IPFS storage area; Storing the index information in a blockchain.
7. A data storage device is applied to a first communication node of the InterPlanetary File System (IPFS) described in any one of claims 1 to 3, characterized in that, The device includes: A first processing module, configured to receive in real time, through the acquisition interface in the IPFS, the log data sent by a second communication node, cache it in the buffer area in the IPFS, and not store it; A second processing module, configured to move the cached log data to the IPFS storage area in the IPFS for storage when the log data cached in the buffer area reaches a preset data volume.
8. The data storage device according to claim 7, wherein It further includes: A first execution module, configured to, after moving the cached log data to the IPFS storage area in the IPFS for storage, return to execute the operation of receiving in real time, through the acquisition interface in the IPFS, the log data sent by a second communication node, caching it in the buffer area in the IPFS, and not storing it.
9. The data storage device according to claim 7, wherein It further includes: A first generation module, configured to generate index information of the log data moved from the buffer area to the IPFS storage area; A first storage module, configured to store the index information in a blockchain.
10. A communication node, the communication node being a first communication node including the InterPlanetary File System (IPFS) according to any one of claims 1 to 3, characterized in that, The communication node includes: a processor and a transceiver; The processor is configured to use the transceiver to receive, in real time, log data sent by a second communication node through the acquisition interface in the IPFS, and cache the log data in a buffer area in the IPFS without storing it; When the log data cached in the buffer area reaches a preset data volume, move the cached log data to an IPFS storage area in the IPFS for storage.
11. A communication node, comprising a memory, a processor, and a program stored on the memory and executable on the processor; characterized in that, When the processor executes the program, it implements the data storage method according to any one of claims 4 to 6.
12. A readable storage medium, on which a program is stored, characterized in that, When the program is executed by the processor, it implements the steps in the data storage method according to any one of claims 4 to 6.
Citation Information
Patent Citations
Log processing method and device based on block chain, computer equipment and storage medium
CN111092745A
Intelligent agricultural data monitoring and automatic control method based on double-chain mechanism
CN111968000A