A log management method of an edge gateway and related equipment

Abstract

The application discloses a log management method of an edge gateway and related equipment, and relates to the technical field of edge gateways. The method comprises the following steps: dividing a cache area, an archiving area and a recycling area in a local storage space of the gateway; obtaining log entries to be stored, writing the log entries into the cache area and generating sequence identifiers; when the cache area meets preset archiving conditions, packing multiple log entries in the cache area into log data blocks, performing digital signature to ensure the integrity, and then storing the log data blocks into the archiving area; when it is detected that a communication link with a cloud server is restored, reading each log data block from the archiving area, sorting all the data blocks according to the sequence identifiers of the log entries in the data blocks, and sequentially uploading the data blocks to the cloud server according to the sorting result; after the uploading is successful, transferring the uploaded log data blocks to the recycling area to release the storage space in the archiving area, and recycling the space in the recycling area for recycling.

Description

Technical Field

[0001] This application relates to the field of edge gateway technology, and in particular to a log management method and related equipment for an edge gateway. Background Technology

[0002] Industrial edge gateways are typically deployed in unstable network environments such as factory workshops and energy sites, frequently encountering network outages or bandwidth limitations. Traditional log management methods often involve directly writing logs to local files, then uploading the entire log file to the cloud via file transfer protocols or hypertext transfer protocols once the network is restored. However, during network outages, local storage space may become full due to continuous log writing, preventing subsequent logs from being stored. Upon recovery, issues such as duplicate log uploads or outdated time sequences can easily occur. Furthermore, local log files lack anti-tampering mechanisms, failing to guarantee that log data has not been maliciously modified or corrupted during network outages, making it difficult to use log data as evidence for incident tracing or compliance audits. Therefore, a new log management method for edge gateways is urgently needed to address the aforementioned technical problems. Summary of the Invention

[0003] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solutions, nor is it intended to determine the scope of protection of the claimed technical solutions.

[0004] Firstly, this application proposes a log management method for an edge gateway, the method comprising:

[0005] Divide the local storage space of the gateway into a cache area, an archive area, and a recycling area;

[0006] Obtain the log entries to be stored, write the log entries into the cache area, and generate a sequence identifier for the log entries, wherein the sequence identifier is used to identify the generation order of the log entries;

[0007] When the cache area meets the preset sealing conditions, multiple log entries in the cache area are packaged into a log data block, the log data block is digitally signed, and the digitally signed log data block is stored in the sealing area.

[0008] When the communication link with the cloud server is detected to be restored, at least one of the log data blocks is read from the archived area;

[0009] Based on the sequence identifier of the log entries in each log data block, the log data blocks are sorted, and the log data blocks are uploaded to the cloud server according to the sorting result.

[0010] After the log data block is successfully uploaded, the uploaded log data block is transferred to the recycling area to release the storage space occupied by the uploaded log data block in the sealing area.

[0011] In some implementations, the sequence identifier is a globally incrementing sequence number or a local timestamp.

[0012] In some implementations, when the cache region meets preset sealing conditions, packaging multiple log entries in the cache region into a log data block includes:

[0013] Obtain the cached data volume or cache duration of log entries in the cache area;

[0014] When the amount of cached data is greater than or equal to a preset data volume threshold, it is determined that the cache region meets the preset sealing condition, and all log entries in the cache region are packaged into a log data block; or...

[0015] When the duration of the cache is greater than or equal to a preset duration threshold, it is determined that the cache area meets the preset sealing condition, and all log entries in the cache area are packaged into the log data block.

[0016] In some implementations, the digital signature processing of the log data block includes:

[0017] Obtain the key information stored in the gateway;

[0018] Based on the key information, a hash operation is performed on each log entry contained in the log data block to generate a digital signature for the log data block.

[0019] In some implementations, sorting the log data blocks according to the sequence identifiers of log entries in each log data block includes:

[0020] For each log data block, the smallest sequence identifier is selected from the sequence identifiers of each log entry contained in the log data block, and used as the base sequence identifier of the log data block.

[0021] The log data blocks are sorted according to the size order of the reference sequence identifiers of each log data block.

[0022] In some implementations, the step of transferring the uploaded log data block to the reclaimed area after the log data block is successfully uploaded includes:

[0023] Receive confirmation information returned by the cloud server for the uploaded log data blocks;

[0024] In response to the confirmation information, the storage location of the uploaded log data block in the archive area is determined;

[0025] Based on the storage location, the uploaded log data block is read from the sealed area and written to the reclaimed area;

[0026] After the uploaded log data block is written to the reclaimed area, the uploaded log data block is deleted from the archived area.

[0027] In some embodiments, the method further includes:

[0028] Obtain the remaining storage capacity of the sealed area;

[0029] When the remaining storage capacity is less than a preset capacity threshold, the log entries to be stored are degraded and recorded.

[0030] In some implementations, the step of performing a downgrade recording operation on the log entries to be stored when the remaining storage capacity is less than a preset capacity threshold includes:

[0031] Obtain the log type of the log entry to be stored, wherein the log type includes alarm type, event type and telemetry type;

[0032] Log entries belonging to the alarm class and the event class are identified as high-priority log entries, and these high-priority log entries are fully recorded.

[0033] Log entries belonging to the telemetry category are identified as low-priority log entries, and a portion of these low-priority log entries are selected for storage based on a preset sampling ratio.

[0034] Secondly, this application proposes a log management device for an edge gateway, the device comprising:

[0035] The region partitioning unit is used to divide the local storage space of the gateway into cache region, archive region and reclaim region;

[0036] A log writing unit is used to obtain log entries to be stored, write the log entries into the cache area, and generate a sequence identifier for the log entries, wherein the sequence identifier is used to identify the generation order of the log entries;

[0037] A data sealing unit is used to package multiple log entries in the cache area into a log data block when the cache area meets the preset sealing conditions, perform digital signature processing on the log data block, and store the digitally signed log data block into the sealing area.

[0038] A data reading unit is configured to read at least one of the log data blocks from the sealed area when a communication link with the cloud server is detected to be restored.

[0039] The sorting and uploading unit is used to sort each log data block according to the sequence identifier of the log entries in each log data block, and upload each log data block to the cloud server according to the sorting result.

[0040] The space reclamation unit is used to transfer the uploaded log data block to the reclamation area after the log data block is successfully uploaded, so as to release the storage space occupied by the uploaded log data block in the sealing area.

[0041] Thirdly, this application proposes an electronic device comprising: a memory, a processor, and a computer program stored in the memory that can run on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of the log management method of the edge gateway according to any of the first aspects.

[0042] In summary, the edge gateway log management method provided in this application divides the gateway's local storage space into a cache area, an archive area, and a recycling area. The cache area handles high-frequency write operations of log entries, temporarily storing logs on the fastest read / write storage medium to avoid congestion during log generation due to slow write speeds. The archive area persistently stores logs, ensuring that log data is not lost due to power outages or system restarts during network outages. The recycling area receives successfully uploaded log data blocks, allowing the storage space in the archive area to be reused. This division of the three areas provides an architecture for log data management on the gateway, from temporary caching to persistent archiving and then space release.

[0043] The system retrieves log entries to be stored, writes them to the buffer area, and generates sequence identifiers for each entry during the writing process. These sequence identifiers are used to mark the order in which the log entries were created. By assigning a sequence identifier to each log entry during the log writing phase, all processing operations based on log order have a data source, avoiding inaccurate order determination caused by dependence on physical storage location.

[0044] When the cache area meets the preset sealing conditions, multiple log entries in the cache area are packaged into a log data block, and the log data block is digitally signed before being stored in the sealing area. By integrating scattered log entries into log data blocks for unified processing, the granularity of data management is reduced, which is beneficial to the scheduling efficiency during batch uploads. At the same time, digital signature processing gives the log data block a verifiable integrity identifier after it is stored in the sealing area. If the log data block is modified or corrupted during storage, it can be identified through digital signature verification.

[0045] When the communication link with the cloud server is detected to be restored, at least one log data block is read from the archived area, and the log data blocks are sorted according to the sequence identifiers of the log entries in each block. Based on the sorting result, each log data block is uploaded to the cloud server. This step first sorts the log data blocks using sequence identifiers before performing the upload operation, ensuring that the order of the log data blocks uploaded to the cloud matches the actual generation order. The cloud does not need to rely on the receiving timestamps for additional order inference or rearrangement, avoiding misjudgments of order due to network transmission latency fluctuations and preventing data redundancy caused by repeatedly uploading the same data block.

[0046] After a log data block is successfully uploaded, it is moved to the reclaimed area to release the storage space occupied by the uploaded log data blocks in the archive area. This step, by removing the successfully uploaded data blocks from the archive area and placing them in the reclaimed area, allows the storage capacity of the archive area to be released again after a network outage, thus continuing to receive newly generated log data. This ensures the sustainable write capability of the archive area during long-term operation and avoids the archive area becoming full and unable to store new logs due to uploaded data occupying storage space for an extended period. Attached Figure Description

[0047] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0048] Figure 1 A schematic diagram of a log management method for an edge gateway provided in an embodiment of this application;

[0049] Figure 2 A schematic diagram of the log management device structure of an edge gateway provided in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of an edge gateway log management electronic device provided in an embodiment of this application. Detailed Implementation

[0051] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or device that includes a series of steps or units, not limited to those explicitly listed, but also including other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. The technical solutions in the embodiments of this application will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0052] This application relates to the field of edge gateway technology, and more particularly to a log management method and related equipment for an edge gateway. An edge gateway is a gateway device deployed at the network edge, connecting the physical and digital worlds through network connectivity, protocol conversion, and other functions, providing real-time data analysis and application management capabilities. In industrial IoT scenarios, the edge gateway serves to connect sensing layer devices with upper-layer communication networks, specifically including protocol conversion, data acquisition, edge computing, and secure encrypted transmission. Edge gateways are typically deployed in complex and dynamic network environments such as factory workshops, energy sites, and smart cities, where communication links between them and cloud servers often face frequent interruptions or bandwidth limitations.

[0053] In such network outage scenarios, the gateway still needs to continue operating locally and generate a large amount of log data. This log data is crucial for equipment maintenance, fault diagnosis, and accident tracing. However, since log data cannot be uploaded to the cloud in real time during network outages, it can only be temporarily stored locally on the gateway and resumed after the network is restored. Therefore, ensuring the integrity of log data storage during network outages, the timeliness of the resumed transmission, and the controllability of storage space have become urgent technical problems to be solved in the field of edge gateway log management. The log management method provided in this application can be applied to various edge gateway devices deployed in unstable network environments, and is particularly suitable for industrial IoT applications such as industrial automation, intelligent manufacturing, and smart energy management, which have high requirements for the integrity, auditability, and timeliness accuracy of log data.

[0054] To clearly describe the technical solution of this application, the meanings of specific terms in this application will be explained below.

[0055] The log entries in this application refer to time-stamped records generated by an industrial edge gateway during operation, based on changes in the status of field devices, event occurrences, data sampling, and other actions. Each log entry contains the specific content recorded in that log entry, as well as metadata information related to that log entry. Log entries are the basic data unit for log management, and multiple log entries can be aggregated into log data blocks for unified management.

[0056] In this application, a log data block refers to an aggregated data unit formed by packaging multiple log entries in the cache area. A log data block is an intermediate-level data management unit between a single log entry and a complete log file; its granularity is larger than a single log entry but smaller than a complete log file. By aggregating scattered log entries into log data blocks, the number of fragmented writes to the storage medium can be reduced, transmission overhead during the upload phase can be lowered, and digital signature processing and storage space management can be facilitated on a block-by-block basis.

[0057] The sequence identifier in this application refers to a tag generated for each log entry to uniquely identify the order in which the log entry was generated. The sequence identifier is assigned when a log entry is generated, and its value is determined based on the actual time of its generation, reflecting the chronological relationship between log entries. The sequence identifier can be a globally incrementing sequence number or a local timestamp; regardless of the form, it restores the actual generation order of each log entry when sorting log data blocks. The sequence identifier is independent of the physical storage location of the log entries; therefore, even if the storage order is disrupted due to multi-threaded concurrent writes or abnormal system restarts, the temporal relationship determined by the sequence identifier remains accurate and reliable.

[0058] The digital signature in this application refers to verification information generated from log data blocks using cryptographic techniques to verify the integrity and authenticity of those data blocks. The generation process typically involves hashing each log entry in the log data block in its ordered sequence to generate a fixed-length hash value. This hash value is then encrypted using key information stored in the gateway security element, forming the digital signature for the log data block. When verifying the integrity of the log data block, the content is re-hashed and compared with the information in the digital signature to determine whether the data block has been tampered with or damaged during storage. This application uses digital signature technology to ensure the integrity and traceability of log data in the sealed area.

[0059] The cache area, sealing area, and recycling area in this application refer to three logical areas divided within the gateway's local storage space. The cache area handles high-frequency write operations for log entries, temporarily storing newly generated log entries in a fast-read / write storage medium. These entries are then transferred out in batches once preset conditions are met, avoiding the frequent erase / write wear caused by writing each log entry individually to persistent storage. The sealing area is used for persistent storage of log data. Log data blocks, after being packaged and digitally signed, are stored in this area, ensuring that log data is not lost due to power outages or system restarts during network interruptions. The recycling area receives log data blocks successfully uploaded to the cloud server, freeing up and reusing the storage space in the sealing area. These three areas work together to form a complete path for log data from temporary caching to persistent sealing and then to space reclamation.

[0060] The preset sealing condition in this application refers to the criteria for determining whether log entries in the cache area are packaged into log data blocks and stored in the sealing area. The specific form of the preset sealing condition can be based on the cumulative data volume of log entries in the cache area, such as triggering sealing when the cached data volume reaches a preset data volume threshold; or it can be based on the cache duration of log entries in the cache area, such as triggering sealing when the cache duration reaches a preset duration threshold. By setting the preset sealing condition, the cache area can be prevented from becoming full due to continuous log writing, while ensuring that log data is transferred from the volatile cache to the persistent sealing area in a timely manner.

[0061] The degraded recording operation in this application refers to a selective recording strategy for log entries to be stored when the remaining storage capacity of the sealed area is insufficient. The purpose of the degraded recording operation is to prioritize the complete recording of important log types when storage resources are limited, while sampling or reducing the frequency of recording for secondary log types to extend the writable duration of the sealed area. For example, alarm and event logs typically involve equipment anomalies or critical status changes and are of high value for fault diagnosis and accident tracing; therefore, they are identified as high-priority logs and recorded in full. Telemetry logs, on the other hand, are periodically collected routine operational data, and the value of a single entry is relatively low; therefore, they are identified as low-priority logs, and a portion of the entries are selected for storage according to a preset sampling ratio. The degraded recording operation serves as an emergency protection mechanism when the sealed area space is insufficient, maximizing the preservation of valuable log information under special circumstances.

[0062] Please see Figure 1 This is a schematic diagram of a log management method for an edge gateway provided in an embodiment of this application, which may specifically include:

[0063] S110. Divide the local storage space of the gateway into a cache area, a storage area, and a recycling area;

[0064] For example, the cache area is located in the storage medium with the fastest read and write speed, used for high-frequency writing of log entries, avoiding the frequent erase and write overhead caused by directly writing each log to persistent storage; the archive area is located in the non-volatile storage medium, used to persistently save the log data blocks that have been packaged and signed, ensuring that the log data is not lost in the event of network interruption or system abnormality; the reclaim area is used to receive the log data blocks that have been successfully uploaded, so that the storage space occupied and uploaded in the archive area can be released, maintaining the continuous writability of the archive area.

[0065] S120. Obtain the log entries to be stored, write the log entries to the buffer area, and generate a sequence identifier for the log entries, wherein the sequence identifier is used to identify the generation order of the log entries;

[0066] For example, the gateway continuously generates log entries during operation, each recording status changes or event information of field devices. After acquiring a log entry to be stored, it writes it to a cache area to leverage the high-speed read / write performance of the cache area for log write operations. Simultaneously with writing to the cache area, a sequence identifier is generated for the log entry. This sequence identifier is assigned when the log entry is generated and is used to mark the order in which the log entries were created. Since the sequence identifier is determined based on the actual time of log entry generation, its value reflects the sequential relationship between log entries and is independent of the physical storage location of the log entries in the storage medium. Therefore, even if multi-threaded concurrent writing or abnormal system restarts disrupt the storage order, the temporal relationship recorded by the sequence identifier remains accurate.

[0067] S130. When the cache area meets the preset sealing conditions, multiple log entries in the cache area are packaged into log data blocks, and the log data blocks are digitally signed and stored in the sealing area.

[0068] For example, log entries continuously written to the cache region accumulate. When the cache region meets preset sealing conditions, the log data is transferred from the cache region to the sealing region. Multiple log entries in the cache region are retrieved and aggregated into a single log data block, integrating scattered log entries into a larger-granularity data management unit. This facilitates unified integrity protection and storage management at the data block level. The log data block undergoes digital signature processing. The digital signature is verification information generated based on cryptographic techniques, used to verify the integrity and authenticity of the data block. The digitally signed log data block is stored in the sealing region, ensuring that the log data is persistently retained in a non-volatile storage medium in an integrity-protected form, guaranteeing that the log data will not be lost during network outages or system anomalies.

[0069] S140. When the communication link with the cloud server is detected to be restored, at least one log data block is read from the archived area;

[0070] For example, the gateway continuously monitors the communication link status between itself and the cloud server. When it detects that the communication link has been restored from an interrupted state to a connected state, it initiates the log data upload process. Log data blocks are read from the archived area as the data source to be uploaded to the cloud server. The number of log data blocks read is determined by the actual number of log data blocks stored in the archived area; one or multiple log data blocks can be read. After reading the log data blocks from the archived area, they can be used as input data for the upload operation.

[0071] S150. Sort each log data block according to the sequence identifier of the log entries in each log data block, and upload each log data block to the cloud server according to the sorting result.

[0072] For example, after reading log data blocks from the archived area, the sequence identifiers carried by the log entries in each log data block are obtained. These sequence identifiers record the actual generation order of each log entry. Based on these sequence identifiers, the log data blocks are sorted so that the order of the sorted log data blocks matches the actual generation order of the log entries within each log data block. According to the sorting result, each log data block is uploaded to the cloud server sequentially. The log data block sequence received by the cloud server is identical to the actual generation order of its corresponding log entries, eliminating the need for additional order deduction or rearrangement by the cloud server.

[0073] S160. After the log data block is successfully uploaded, the uploaded log data block is transferred to the recycling area to release the storage space occupied by the uploaded log data block in the archive area.

[0074] For example, after a log data block is uploaded to the cloud server, the gateway receives a successful upload confirmation from the cloud server, confirming that the log data block has been successfully received by the cloud. Based on this confirmation, the uploaded log data block is transferred from the archive area to the reclaim area. This transfer operation removes the uploaded log data block from the archive area, releasing the storage space occupied by these data blocks and increasing the remaining storage capacity of the archive area. This allows the archive area to continue receiving newly generated log data blocks, ensuring its sustainable write capability during long-term operation. The log data blocks transferred to the reclaim area are isolated from the log data blocks awaiting upload in the archive area and will not affect the normal management and uploading of other log data blocks in the archive area.

[0075] In summary, the edge gateway log management method provided in this application divides the gateway's local storage space into a cache area, an archive area, and a recycling area. The cache area is used to handle high-frequency write operations of log entries, temporarily storing logs in the fastest read / write storage medium to avoid congestion during log generation due to slow writes. The archive area is used to persistently save logs, ensuring that log data is not lost due to power outages or system restarts during network outages. The recycling area is used to receive successfully uploaded log data blocks, allowing the storage space of the archive area to be reused. Through the division of the above three areas, an architecture is provided for log data in the gateway's local storage, from temporary caching to persistent archiving and then to space release.

[0076] The system retrieves log entries to be stored, writes them to the buffer area, and generates sequence identifiers for each entry during the writing process. These sequence identifiers are used to mark the order in which the log entries were created. By assigning a sequence identifier to each log entry during the log writing phase, all processing operations based on log order have a data source, avoiding inaccurate order determination caused by dependence on physical storage location.

[0077] When the cache area meets the preset sealing conditions, multiple log entries in the cache area are packaged into a log data block, and the log data block is digitally signed before being stored in the sealing area. By integrating scattered log entries into log data blocks for unified processing, the granularity of data management is reduced, which is beneficial to the scheduling efficiency during batch uploads. At the same time, digital signature processing gives the log data block a verifiable integrity identifier after it is stored in the sealing area. If the log data block is modified or corrupted during storage, it can be identified through digital signature verification.

[0078] When the communication link with the cloud server is detected to be restored, at least one log data block is read from the archived area, and the log data blocks are sorted according to the sequence identifiers of the log entries in each block. Based on the sorting result, each log data block is uploaded to the cloud server. This step first sorts the log data blocks using sequence identifiers before performing the upload operation, ensuring that the order of the log data blocks uploaded to the cloud matches the actual generation order. The cloud does not need to rely on the receiving timestamps for additional order inference or rearrangement, avoiding misjudgments of order due to network transmission latency fluctuations and preventing data redundancy caused by repeatedly uploading the same data block.

[0079] After a log data block is successfully uploaded, it is moved to the reclaimed area to release the storage space occupied by the uploaded log data blocks in the archive area. This step, by removing the successfully uploaded data blocks from the archive area and placing them in the reclaimed area, allows the storage capacity of the archive area to be released again after a network outage, thus continuing to receive newly generated log data. This ensures the sustainable write capability of the archive area during long-term operation and avoids the archive area becoming full and unable to store new logs due to uploaded data occupying storage space for an extended period.

[0080] In some embodiments, the sequence identifier is a globally incrementing sequence number or a local timestamp.

[0081] For example, the sequence identifier can be a globally incrementing sequence number or a local timestamp. For example, when a globally incrementing sequence number is used, the gateway assigns the current sequence number to each log entry when generating it, and then increments the sequence number to ensure that the sequence number value of each log entry strictly increases. In other embodiments, when a local timestamp is used, the gateway obtains the current time value of the local system time as the timestamp of each log entry when generating it. Regardless of whether a globally incrementing sequence number or a local timestamp is used, the sequence relationship between log entries can be reflected, and this relationship is determined at the log entry generation stage and is not affected by subsequent changes in storage location.

[0082] In some embodiments, when a cache region meets preset sealing conditions, multiple log entries in the cache region are packaged into a log data block, including:

[0083] Get the amount of cached data or the duration of the cache for log entries in the cache area;

[0084] When the amount of cached data is greater than or equal to a preset data volume threshold, the cache area is determined to meet the preset sealing conditions, and all log entries in the cache area are packaged into a log data block; or...

[0085] When the cache duration is greater than or equal to the preset duration threshold, the cache area is determined to meet the preset sealing conditions, and all log entries in the cache area are packaged into log data blocks.

[0086] For example, during operation, the gateway continuously monitors the accumulation status of log entries in the cache region and obtains the cached data volume of log entries in the cache region. When the cached data volume reaches a preset data volume threshold, it indicates that a sufficient number of log entries have accumulated in the cache region, and at this time, the cache region is determined to meet the preset sealing conditions. Alternatively, the gateway obtains the cache duration of log entries in the cache region. This cache duration reflects the time span from the earliest written log entry in the cache region to the current time. When the cache duration reaches a preset duration threshold, even if the cached data volume has not yet reached the preset data volume threshold, the cache region is still determined to meet the preset sealing conditions. By setting two different triggering methods, the sealing operation can be performed in batches when the log accumulation volume is large, and can also prevent logs from remaining in the volatile cache region for a long time when the log generation rate is low.

[0087] Once the cache region meets the preset sealing conditions, the gateway retrieves all currently stored log entries from the cache region and aggregates these scattered log entries into a single log data block. This log data block then serves as an independent data management unit for write operations within the sealing region. By packaging all log entries in the cache region into a single log data block, each sealing operation generates one log data block, and the log data blocks are arranged in a temporal sequence according to the order in which the sealing operations are triggered.

[0088] In summary, the high-frequency write operations of log entries in the cache area of ​​this application embodiment have limited storage space. If the log entries are not archived and transferred out for a long period of time, the cache area will be filled with continuously written log entries and will be unable to receive newly generated log entries. By setting a cache data volume threshold as an archiving trigger condition, the gateway immediately performs package archiving when the cache data volume reaches the threshold, transferring the log entries in the cache area in batches. After the transfer is completed, the storage space occupied by the log entries that have been retrieved from the cache area is released, and the used storage space of the cache area drops below the threshold, so that there is always available space to receive new log entries.

[0089] On the other hand, in scenarios with low log generation rates, the amount of cached data may not reach the preset data volume threshold for an extended period. If only the data volume threshold is relied upon to trigger sealing, log entries will remain in the cache area for a long time. Since the cache area is typically deployed in volatile storage media, all log entries remaining in the cache area will be lost in the event of a system restart or power outage. By setting a cache duration threshold as a supplementary trigger condition, sealing and transferring logs out are triggered even if the data volume is not yet large enough when the cache duration reaches this threshold. This promptly transfers the small amount of logs remaining in the cache area to the non-volatile sealing area, preventing log entries from being lost in the event of a system anomaly due to prolonged retention in the volatile cache. All log entries in the cache area are packaged into a log data block. Each sealing operation generates a complete log sequence fragment. The log data blocks form a temporal relationship according to the order in which sealing is triggered. During subsequent upload phases, when data blocks are transmitted, the order between the data blocks is determined by the sealing trigger order, eliminating the need to establish additional sequential association information between data blocks.

[0090] In some embodiments, digital signature processing of log data blocks includes:

[0091] Retrieve the key information stored in the gateway;

[0092] Based on the key information, a hash operation is performed on each log entry contained in the log data block to generate a digital signature for the log data block.

[0093] For example, the gateway reads pre-stored key information from local storage. The gateway obtains a pre-packaged log data block containing multiple log entries arranged sequentially within the block. The gateway performs a hash operation on each log entry in the log data block according to their order of arrangement. It then reads the content data of each log entry sequentially, combines the content data of each entry into a complete data sequence according to their order within the log data block, and performs a hash operation on this data sequence to generate a fixed-length hash value.

[0094] A hash operation transforms input data of arbitrary length into a fixed-length output value. Any small change in the input data will cause a significant change in the output value; therefore, the hash value uniquely represents the combined content state of all log entries in the log data block. The gateway performs an encryption operation on this hash value based on the acquired key information, and uses the ciphertext result as the digital signature of the log data block. This digital signature is associated with the content of the log data block and is stored in the sealing area along with the log data block.

[0095] In summary, this application embodiment obtains key information stored in the gateway and performs hash operations on each log entry contained in the log data block based on the key information to generate a digital signature, thus establishing a fixed mathematical binding relationship between the digital signature and each log entry in the log data block. When it is necessary to verify the integrity of the log data block stored in the sealed area, the log entries in the log data block are recombined in the same order and hashed to obtain a new hash value. Then, based on the same key information, an encryption operation is performed on the new hash value to obtain a new digital signature. By comparing the new digital signature with the original digital signature stored when it was stored in the sealed area, it can be determined whether the log data block has been modified during its storage in the sealed area.

[0096] If two digital signatures match, it indicates that the content of each log entry in the log data block has not been altered during storage; if the two digital signatures do not match, it indicates that the content in the log data block has been tampered with or the data has been corrupted. Only the gateway holding the correct key can generate a valid digital signature. External devices cannot forge a valid digital signature without the key information, ensuring the unforgeability of digital signatures. Digital signatures are stored in association with log data blocks in the archive area, giving each log data block stored in the archive area an independently verifiable integrity identifier. Even if the archive area is exposed to an insecure environment for a long time, whether the log data block has been tampered with can be accurately identified through signature comparison, providing a verifiable integrity protection mechanism for the log data stored in the archive area.

[0097] In some embodiments, the log data blocks are sorted according to the sequence identifiers of log entries in each log data block, including:

[0098] For each log data block, the smallest sequence identifier is selected from the sequence identifiers of each log entry contained in the log data block, and this is used as the base sequence identifier of the log data block.

[0099] The log data blocks are sorted according to the size order of their base sequence identifiers.

[0100] For example, each log data block read from the archived area contains multiple log entries, each carrying a sequence identifier, and the sequence identifiers of different log entries within the same log data block have different values. The gateway performs a baseline sequence identifier selection operation on each log data block: it iterates through all log entries contained in the log data block, sequentially reads the sequence identifier of each log entry, compares the currently read sequence identifier with the temporarily stored minimum value, and replaces it if the current value is less than the temporarily stored value. After the iteration is complete, the temporarily stored value is the minimum sequence identifier among all sequence identifiers in the log data block, and this minimum sequence identifier is determined as the baseline sequence identifier for the log data block. After performing the above operation on each log data block, each log data block corresponds to a baseline sequence identifier.

[0101] After obtaining the baseline sequence identifiers for all log data blocks, the gateway sorts the log data blocks according to the numerical value of the baseline sequence identifiers. Log data blocks with smaller baseline sequence identifier values ​​are arranged first, and those with larger values ​​are arranged last. The sorted order of the log data blocks corresponds to the chronological order of their earliest generated log entries. This sorting method determines the temporal relationship between data blocks based on the generation order of the earliest log entries within each block. Since the sequence identifiers are assigned when log entries are generated, reflecting the actual generation order of each log entry, it is not affected by the physical location of the storage medium or the file system's arrangement order, thus ensuring accurate and reliable sorting results.

[0102] In some embodiments, after a log data block is successfully uploaded, the uploaded log data block is transferred to the recycling area, including:

[0103] Receive confirmation information from the cloud server for the uploaded log data blocks;

[0104] In response to the confirmation message, determine the storage location of the uploaded log data blocks in the archive area;

[0105] Based on the storage location, the uploaded log data blocks are read from the archive area and written to the reclaim area.

[0106] After the uploaded log data blocks are written to the reclaim area, the uploaded log data blocks in the archive area are deleted.

[0107] For example, after a log data block is uploaded to the cloud server, the cloud server performs an integrity check on the received log data block. If the check passes, the cloud server returns an confirmation message to the gateway, indicating that the log data block has been successfully received and verified by the cloud. Upon receiving this confirmation message, the gateway determines the log data block corresponding to that confirmation message as a successfully uploaded data block.

[0108] The gateway, based on the storage management records of the archived area, locates the storage location information of the uploaded log data block within the archived area. This storage location information includes the starting address and occupied length of the log data block within the archived area. Based on this storage location information, the gateway reads the complete data content of the log data block from the corresponding address in the archived area and writes the read log data block to the reclaimed area. After confirming that the log data block has been completely written to the reclaimed area, the gateway marks the storage space occupied by the log data block in the archived area as reclaimable and deletes the storage record of the log data block from the archived area.

[0109] In summary, this application ensures that only log data blocks that have been successfully received by the cloud are removed from the archive area by receiving confirmation information returned by the cloud server and using this confirmation information as the trigger condition for the transfer operation. Unconfirmed log data blocks are retained in the archive area and can still be read and uploaded again when the network is interrupted again, thus avoiding data loss.

[0110] Before deleting a log data block in the archive area, the entire log data block is written to the reclaim area. This ensures that a copy of the log data block remains in the reclaim area even after deletion. Even if the cloud subsequently detects a data transmission anomaly and needs to retrieve the data block again, the gateway can still extract it from the reclaim area for processing. After deleting the log data block in the archive area, the storage space occupied by that data block is released, increasing the remaining storage capacity of the archive area to continue receiving newly generated log data blocks, ensuring the archive area's sustainable write capability during long-term operation. The reclaim area and the archive area are independent of each other. Uploaded data blocks stored in the reclaim area are isolated from data blocks to be uploaded stored in the archive area. Data deletion operations are only performed in the archive area, without affecting data already stored in the reclaim area or interfering with other log data blocks to be uploaded in the archive area. Through this process, the safe release of storage space in the archive area is achieved while ensuring data traceability.

[0111] In some embodiments, the method further includes:

[0112] Obtain the remaining storage capacity of the sealed area;

[0113] When the remaining storage capacity is less than a preset capacity threshold, a downgrade recording operation is performed on the log entries to be stored, including:

[0114] Retrieve the log type of the log entry to be stored, where the log type includes alarm, event, and telemetry.

[0115] Log entries belonging to the alarm and event categories are identified as high-priority log entries, and high-priority log entries are fully recorded.

[0116] Log entries belonging to the telemetry category are identified as low-priority log entries, and a portion of these low-priority log entries are selected for storage based on a preset sampling ratio.

[0117] For example, during operation, the gateway continuously monitors the remaining storage capacity of the archive area. This remaining storage capacity gradually decreases as new log data blocks are continuously written to the archive area, and gradually increases as uploaded log data blocks are transferred to the reclaim area. The gateway compares the obtained remaining storage capacity with a preset capacity threshold, which is a pre-set space margin warning value used to trigger a degradation recording operation. When the remaining storage capacity is greater than or equal to the preset capacity threshold, it indicates that the archive area still has enough storage space to accommodate subsequent log data blocks, and the gateway stores the log entries completely according to the normal recording method. When the remaining storage capacity is less than the preset capacity threshold, it indicates that the available storage space in the archive area is insufficient. If the normal recording method is continued to store all types of log entries completely, the storage space in the archive area will soon be exhausted, causing subsequent log entries to be unable to be stored in the archive area and to be lost. At this time, the gateway triggers a degradation recording operation.

[0118] After the gateway triggers a degradation logging operation, it obtains the log type of the currently stored log entry. This log type is marked by the gateway based on the nature of the recorded content when the log entry is generated, including alarm, event, and telemetry logs. Alarm logs record abnormal conditions or fault information that occur during device operation, while event logs record changes in device status or information about critical operations. These two types of logs are directly related to fault diagnosis and accident tracing, and their completeness and continuity directly affect the accuracy of problem localization. The gateway identifies log entries belonging to the alarm and event categories as high-priority log entries and records them completely according to the normal recording method; that is, every alarm and event log entry is completely saved to the archive area.

[0119] Telemetry logs record the operating parameters and sampling data periodically collected by the device. While individual log entries are small in size, they are generated frequently. All log entries are saved when storage space is sufficient, but their value is relatively low when storage space is limited. The gateway identifies telemetry log entries as low-priority log entries and selects a subset of these low-priority log entries for storage based on a preset sampling ratio.

[0120] In summary, this embodiment monitors the remaining storage capacity of the sealed area and triggers a degradation recording operation when it falls below a preset capacity threshold. This allows the gateway to adjust its recording strategy when the available space in the sealed area is insufficient, preventing newly generated log entries from being unable to be written due to storage space exhaustion and avoiding the complete loss of log data. By classifying log entries to be stored according to log type, alarm and event logs are given high priority and recorded completely. This ensures that log data of critical value for equipment operation and maintenance and fault diagnosis is completely preserved when storage space is limited. Telemetry logs are given low priority and a portion of log entries are selected for storage according to a preset sampling ratio. This reduces the actual number of telemetry log entries stored while preserving the overall trend of telemetry data changes, lowering the rate at which telemetry logs occupy the storage space of the sealed area per unit time. This slows down the process of the sealed area running out of space and extends the writable time of the sealed area during network outages. The combination of full recording of high-priority logs and sample recording of low-priority logs enables differentiated allocation of storage resources when there is insufficient space in the archive area. This allows limited space to be used primarily for high-value logs, preserving critical log information during network outages to the greatest extent possible.

[0121] The technical solution of this application will be further described in detail below through specific embodiments.

[0122] In a specific application scenario of an energy management gateway in a factory, the gateway uses an ARM-based system-on-a-chip (SoC) as the main control processor, runs a Linux operating system, and uses an embedded multimedia card as the storage medium. Within the gateway's storage space, a cache area is allocated to the high-speed read / write partition of the embedded multimedia card, a sealed area is allocated to a dedicated storage partition of the embedded multimedia card, and a reclaim area is allocated to an independent storage partition of the embedded multimedia card. Specifically, the cache area is allocated 64MB of random access memory for handling high-frequency log entry writing; the sealed area is allocated 512MB of embedded multimedia card storage for persistently storing packaged and digitally signed log data blocks; and the reclaim area is allocated 128MB of embedded multimedia card storage for receiving log data blocks successfully uploaded to the cloud server. The factory's energy management gateway generates approximately 100 log entries per second. At this log generation rate, the sealed area can provide approximately 6 hours of continuous recording capability during network outages. Once the network is restored, the gateway uploads each log data block to the cloud server according to the sorting order, at a rate of approximately 5,000 log entries per second. Alarm and event logs are given the highest priority during the upload process to ensure that critical logs are uploaded to the cloud first. In the actual operation of this embodiment, no logs were lost due to storage space exhaustion in the archived area. The sequence of log data blocks received by the cloud server was consistent with the actual generation order of the log entries within each log data block, and there were no log order errors or duplicate uploads.

[0123] Please see Figure 2 This is a schematic diagram of the structure of a log management device for an edge gateway provided in an embodiment of this application, which may specifically include:

[0124] The region partitioning unit 21 is used to partition the local storage space of the gateway into a cache region, a sealed region, and a reclaim region;

[0125] The log writing unit 22 is used to obtain log entries to be stored, write log entries to the buffer area, and generate sequence identifiers for log entries. The sequence identifiers are used to identify the generation order of log entries.

[0126] The data sealing unit 23 is used to package multiple log entries in the cache area into a log data block when the cache area meets the preset sealing conditions, and to perform digital signature processing on the log data block and store the digitally signed log data block into the sealing area.

[0127] The data reading unit 24 is used to read at least one log data block from the archived area when the communication link with the cloud server is detected to be restored.

[0128] The sorting and uploading unit 25 is used to sort each log data block according to the sequence identifier of the log entries in each log data block, and upload each log data block to the cloud server according to the sorting result.

[0129] Space reclamation unit 26 is used to transfer the uploaded log data blocks to the reclamation area after the log data blocks are successfully uploaded, so as to release the storage space occupied by the uploaded log data blocks in the archive area.

[0130] Please see Figure 3 This application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and running on the processor. When the processor 320 executes the computer program 311, it implements the steps of the log management method of the edge gateway.

[0131] Since the electronic device described in this embodiment is the device used to implement the log management device of the edge gateway in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application falls within the scope of protection of this application.

[0132] In practice, when the computer program 311 is executed by the processor, it can implement any of the embodiments corresponding to the first aspect.

[0133] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0134] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0135] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications beyond the scope of this specification.

[0136] Obviously, those skilled in the art can make various modifications to this specification without departing from its spirit and scope. Therefore, this specification also intends to include any modifications that fall within the scope of the claims and their equivalents.

Claims

1. A log management method for an edge gateway, characterized in that, include: Divide the local storage space of the gateway into a cache area, an archive area, and a recycling area; Obtain the log entries to be stored, write the log entries into the cache area, and generate a sequence identifier for the log entries, wherein the sequence identifier is used to identify the generation order of the log entries; When the cache area meets the preset sealing conditions, multiple log entries in the cache area are packaged into a log data block, the log data block is digitally signed, and the digitally signed log data block is stored in the sealing area. When the communication link with the cloud server is detected to be restored, at least one of the log data blocks is read from the archived area; Based on the sequence identifier of the log entries in each log data block, the log data blocks are sorted, and the log data blocks are uploaded to the cloud server according to the sorting result. After the log data block is successfully uploaded, the uploaded log data block is transferred to the recycling area to release the storage space occupied by the uploaded log data block in the sealing area.

2. The method according to claim 1, characterized in that, The sequence identifier is either a globally incrementing sequence number or a local timestamp.

3. The method according to claim 1, characterized in that, When the cache region meets the preset sealing conditions, multiple log entries in the cache region are packaged into a log data block, including: Obtain the cached data volume or cache duration of log entries in the cache area; When the amount of cached data is greater than or equal to a preset data volume threshold, it is determined that the cache region meets the preset sealing condition, and all log entries in the cache region are packaged into a log data block; or... When the duration of the cache is greater than or equal to a preset duration threshold, it is determined that the cache area meets the preset sealing condition, and all log entries in the cache area are packaged into the log data block.

4. The method according to claim 1, characterized in that, The digital signature processing of the log data block includes: Obtain the key information stored in the gateway; Based on the key information, a hash operation is performed on each log entry contained in the log data block to generate a digital signature for the log data block.

5. The method according to claim 1, characterized in that, The step of sorting the log data blocks according to the sequence identifier of the log entries in each log data block includes: For each log data block, the smallest sequence identifier is selected from the sequence identifiers of each log entry contained in the log data block, and used as the base sequence identifier of the log data block. The log data blocks are sorted according to the size order of the reference sequence identifiers of each log data block.

6. The method according to claim 1, characterized in that, After the log data block is successfully uploaded, transferring the uploaded log data block to the recycling area includes: Receive confirmation information returned by the cloud server for the uploaded log data blocks; In response to the confirmation information, the storage location of the uploaded log data block in the archive area is determined; Based on the storage location, the uploaded log data block is read from the sealed area and written to the reclaimed area; After the uploaded log data block is written to the reclaimed area, the uploaded log data block is deleted from the archived area.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the remaining storage capacity of the sealed area; When the remaining storage capacity is less than a preset capacity threshold, the log entries to be stored are degraded and recorded.

8. The method according to claim 7, characterized in that, When the remaining storage capacity is less than a preset capacity threshold, the step of performing a downgrade recording operation on the log entries to be stored includes: Obtain the log type of the log entry to be stored, wherein the log type includes alarm type, event type and telemetry type; Log entries belonging to the alarm class and the event class are identified as high-priority log entries, and these high-priority log entries are fully recorded. Log entries belonging to the telemetry category are identified as low-priority log entries, and a portion of these low-priority log entries are selected for storage based on a preset sampling ratio.

9. A log management device for an edge gateway, characterized in that, The device includes: The region partitioning unit is used to divide the local storage space of the gateway into cache region, archive region and reclaim region; A log writing unit is used to obtain log entries to be stored, write the log entries into the cache area, and generate a sequence identifier for the log entries, wherein the sequence identifier is used to identify the generation order of the log entries; A data sealing unit is used to package multiple log entries in the cache area into a log data block when the cache area meets the preset sealing conditions, perform digital signature processing on the log data block, and store the digitally signed log data block into the sealing area. A data reading unit is configured to read at least one of the log data blocks from the sealed area when a communication link with the cloud server is detected to be restored. The sorting and uploading unit is used to sort each log data block according to the sequence identifier of the log entries in each log data block, and upload each log data block to the cloud server according to the sorting result. The space reclamation unit is used to transfer the uploaded log data block to the reclamation area after the log data block is successfully uploaded, so as to release the storage space occupied by the uploaded log data block in the sealing area.

10. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program stored in the memory, implements the steps of the log management method for an edge gateway as described in any one of claims 1 to 8.

Images