EasyLogger-based log management and SFTP automatic synchronization method
By adding color-coded identifiers and generating hash values for logs in embedded devices, combined with automatic upload and local cleanup mechanisms, the problems of traffic and storage in log management are solved, achieving efficient log management and resource saving.
Patent Information
- Application Number
- CN202511752618.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-26
- Publication Date
- 2026-03-17
AI Technical Summary
In embedded devices with traffic metering such as 4G/5G, existing log management solutions suffer from problems such as inconvenient log viewing, untimely uploads, and high traffic consumption. Furthermore, EasyLogger lacks an automated linkage mechanism with remote storage.
The log management method based on EasyLogger is adopted. By adding color indicators to logs of different levels, generating hash values for rolling storage, and automatically uploading to the SFTP server when trigger conditions are met, the local file is deleted after completing file integrity verification. Combining compression and filtering strategies optimizes traffic usage.
It significantly improves log readability and device storage space utilization, maximizing savings in bandwidth and storage space, and is suitable for resource-constrained IoT scenarios.
Smart Images
Figure CN121691304A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of log management technology, specifically a log management and SFTP automatic synchronization method based on EasyLogger. Background Technology
[0002] In embedded devices with data-metered billing systems such as 4G / 5G (e.g., IoT terminals, industrial gateways), continuous log generation consumes significant storage and network resources. Existing solutions typically employ local circular storage or manual uploading, which suffers from inconvenient log viewing, untimely uploads, and high data consumption. While EasyLogger provides efficient logging, it lacks an automated linkage mechanism with remote storage. Therefore, overcoming these technical problems and shortcomings is a key issue that needs to be addressed. Summary of the Invention
[0003] To overcome the aforementioned problems in the prior art, this application provides a method, which adopts the following technical solution:
[0004] Firstly, this application provides a log management and SFTP automatic synchronization method based on EasyLogger, the method comprising:
[0005] When the device is running, the application outputs logs through the integrated EasyLogger component, adds color indicators to the output logs of different levels based on preset rules, and obtains colored logs;
[0006] Based on a preset strategy, the coloring logs are stored in local files in a rolling manner, and a unique hash value is generated for the coloring logs stored in local files;
[0007] Continuously monitor the preset trigger conditions, and when the trigger conditions are met, start the SFTP transfer client to upload the colored logs;
[0008] Once the coloring log is uploaded, the remote server returns a confirmation of receipt and verifies the integrity of the uploaded file based on the hash value of the coloring log.
[0009] Once the uploaded file integrity verification passes, the corresponding log file that has been successfully uploaded to the device will be automatically deleted, freeing up local storage space.
[0010] Furthermore, based on preset rules, color indicators are added to output log information at different levels, which are then applied to terminal display and writing to local log files.
[0011] Furthermore, the hash value is stored along with the coloring log as an identifier for the coloring log stored in the local file.
[0012] Furthermore, each hash value serves as an upload verification identifier.
[0013] Furthermore, upload levels can be set based on different color codes.
[0014] Furthermore, continuously monitor preset trigger conditions. When the trigger conditions are met, start the SFTP transfer client to upload the colored logs, including:
[0015] By integrating the libssh library, a secure connection is established with a remote SFTP server, and the colored log file to be uploaded is transferred to a specified directory on the server.
[0016] Furthermore, before uploading the coloring log, the coloring log is compressed before uploading.
[0017] Furthermore, when colored logs are uploaded, based on the configured upload level, colored logs that do not meet the preset threshold are filtered out, and colored logs that meet the preset threshold are uploaded, thus filtering out low-value logs.
[0018] Furthermore, during periods of weak signal or high data usage, uploads can be delayed or only critical logs can be cached.
[0019] Furthermore, if the integrity verification of the uploaded file fails, a re-upload mechanism is triggered.
[0020] Secondly, this application also provides a log management and SFTP automatic synchronization system based on EasyLogger, the system comprising:
[0021] The colorized log acquisition module is used when the device is running. The application outputs logs through the integrated EasyLogger component and adds color indicators to the output logs of different levels based on preset rules to obtain the colorized logs.
[0022] The hash value generation module is used to store the coloring logs in local files in a rolling manner based on a preset strategy, and to generate a unique hash value for the coloring logs stored in the local files;
[0023] The coloring log upload module is used to continuously monitor preset trigger conditions. When the trigger conditions are met, the SFTP transfer client is started to upload the coloring log.
[0024] The integrity verification module is used to verify the integrity of the uploaded file based on the hash value of the coloring log after the remote server returns a confirmation of receipt following the completion of the coloring log upload.
[0025] The local storage space release module is used to automatically delete the corresponding log files that have been successfully uploaded to the device and release local storage space after the integrity verification of the uploaded file passes.
[0026] Thirdly, this application provides an electronic device, comprising:
[0027] One or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the method as described in the first aspect.
[0028] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect.
[0029] Fifthly, this application provides a computer program that, when executed by a computer, performs the method described in the first aspect.
[0030] In one possible design, the program in the fifth aspect can be stored wholly or partially on a storage medium packaged with the processor, or it can be stored wholly or partially on a memory not packaged with the processor.
[0031] This application has the following beneficial effects:
[0032] 1. This application integrates log coloring, SFTP-based automatic upload, and post-upload local cleanup mechanisms into the classic EasyLogger framework to build a complete log management closed loop. This application can significantly improve log readability and maximize the saving of device traffic and storage space through intelligent log management, selective compression upload, and upload-and-delete strategies. It is suitable for IoT scenarios that are sensitive to cost and resources.
[0033] 2. This application constructs an automated closed-loop management system based on log coloring, cache marking, condition triggering, secure upload, and verification cleanup. It deeply integrates the readability enhancement features of the lightweight logging framework EasyLogger with the SFTP secure file transfer capability based on libssh. Through the "upload and delete" mechanism and configurable traffic optimization strategies, it solves the pain points of log management in resource-constrained embedded devices.
[0034] 3. This application provides a low-traffic, high-efficiency log management solution, which enhances readability through log coloring and utilizes libssh to achieve automated SFTP upload and local cleanup of logs, thereby maximizing the saving of device traffic. Attached Figure Description
[0035] Figure 1This is a schematic diagram illustrating the architecture and workflow of the log management and SFTP automatic synchronization method based on EasyLogger, as described in an embodiment of this application.
[0036] Figure 2 This is a flowchart of the log management and SFTP automatic synchronization method based on EasyLogger according to an embodiment of this application;
[0037] Figure 3 This is a schematic diagram of the log management and SFTP automatic synchronization system based on EasyLogger, as described in an embodiment of this application. Detailed Implementation
[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0039] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0040] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0041] Please refer to Figure 1 The flowchart below illustrates a log management and SFTP automatic synchronization method based on EasyLogger, as provided in this embodiment of the application. The specific implementation process is as follows:
[0042] Step S1: When the device is running, the application outputs logs through the integrated EasyLogger component; color-coded logs are added to different levels of output logs based on preset rules to obtain colored logs.
[0043] It should be noted that the equipment operation is mainly aimed at embedded devices with traffic billing such as 4G / 5G, such as IoT terminals and industrial gateways.
[0044] In this embodiment, color-coded identifiers are added to output log information at different levels based on preset rules, which are then applied to terminal display and writing to local log files, greatly improving the debugging and troubleshooting efficiency of developers.
[0045] It should be noted that color-coded identifiers are added to output logs of different levels based on preset rules. These color identifiers can be set to red, yellow, green, etc., with ERROR being red, WARN being yellow, and INFO being green. These identifiers are displayed through the terminal or by coloring the log file header, thereby improving the readability of the logs.
[0046] Step S2: Based on a preset strategy, the coloring logs are stored in local files in a rolling manner, and a unique hash value is generated for the coloring logs stored in the local files.
[0047] It should be noted that color logs are stored in local files in a rolling manner based on a preset strategy. The preset strategy can be used to perform rolling storage according to log level, log generation time, log size, etc. The specific storage method can be set according to needs.
[0048] In this embodiment, the hash value is stored as an identity identifier for the coloring log in a local file, and is saved together with the coloring log.
[0049] In this embodiment of the application, each hash value serves as an upload verification identifier.
[0050] Step S3: Continuously monitor the preset trigger conditions. When the trigger conditions are met, start the SFTP transfer client to upload the colored log.
[0051] In this embodiment, preset trigger conditions are continuously monitored. When the trigger conditions are met, the SFTP transfer client is started to upload the colored logs, including:
[0052] By integrating the libssh library, a secure connection is established with a remote SFTP server, and the colored log file to be uploaded is transferred to a specified directory on the server.
[0053] SFTP automatic upload includes:
[0054] This application establishes a secure connection to a remote SFTP server based on the libssh library.
[0055] Set trigger conditions to automatically push the colored log files to be uploaded to the specified directory of the SFTP server. Trigger conditions include the log files reaching a preset size, scheduled tasks, or specific log events.
[0056] After the color log file is uploaded, the remote server returns a confirmation message, and the client verifies the file integrity using a hash value.
[0057] In this embodiment of the application, before uploading the coloring log, the coloring log is compressed to reduce the transmission size.
[0058] In this embodiment of the application, when colored logs are uploaded, colored logs that do not meet the preset threshold are filtered according to the configured upload level, and colored logs that meet the preset threshold are uploaded, thus filtering out low-value logs.
[0059] In this application implementation, during periods of weak signal or high data usage, uploads can be delayed or only critical logs can be cached.
[0060] This application optimizes traffic strategies during the upload process by compressing, filtering, and delaying the upload of colored logs or caching only critical logs.
[0061] Step S4: After the coloring log is uploaded, the remote server returns a receipt confirmation and verifies the integrity of the uploaded file based on the hash value of the coloring log.
[0062] It should be noted that this application verifies the integrity of the uploaded files to ensure that the log data is not damaged or lost during transmission.
[0063] Step S5: After the uploaded file integrity verification passes, the corresponding log file that has been successfully uploaded on the device is automatically deleted to free up local storage space.
[0064] In this embodiment of the application, when the integrity verification of the uploaded file fails, a re-upload mechanism is triggered to ensure the reliability of the data.
[0065] Example 2 is a log management and SFTP automatic synchronization system based on EasyLogger provided in this application. The system includes:
[0066] The color-coded log acquisition module 301 is used when the device is running. The application outputs logs through the integrated EasyLogger component and adds color indicators to the output logs of different levels based on preset rules to acquire color-coded logs.
[0067] The hash value generation module 302 is used to store the coloring logs in a local file in a rolling manner based on a preset strategy, and to generate a unique hash value for the coloring logs stored in the local file.
[0068] The coloring log upload module 303 is used to continuously monitor preset trigger conditions. When the trigger conditions are met, the SFTP transfer client is started to upload the coloring log.
[0069] The integrity verification module 304 is used to verify the integrity of the uploaded file based on the hash value of the color log after the remote server returns a receipt confirmation after the color log upload is completed.
[0070] The local storage space release module 305 is used to automatically delete the corresponding log files that have been successfully uploaded to the device and release local storage space after the integrity verification of the uploaded file passes.
[0071] Based on the application scenarios of Embodiment 1 and Embodiment 2 above:
[0072] Application Scenario 1: Monitoring of Internet of Things (IoT) terminal devices;
[0073] Scope of application: Environmental monitoring sensors (such as temperature, humidity, and water quality sensors) deployed in the field or irrigation controllers for smart agriculture, transmitting data back via 4G / 5G networks. The equipment operates unattended for extended periods, and its operational logs are crucial for diagnosing equipment malfunctions and analyzing on-site problems.
[0074] Application of the invention: The device generates color-coded logs locally, allowing on-site technicians to quickly view critical errors via serial port. Simultaneously, when network conditions are good, the system automatically compresses and uploads accumulated error (ERROR) and warning (WARN) logs to the cloud-based operations and maintenance center. After successful upload, it automatically cleans up local logs, significantly saving expensive data traffic on outdoor IoT cards and limited device storage space.
[0075] Application Scenario 2: Industrial gateways and edge computing devices;
[0076] Scope of application: In smart factories, industrial gateways are responsible for collecting and initially processing data from PLCs and sensors on multiple production lines. The stability of the gateway directly affects production, and its logs are crucial for analyzing network jitter and protocol conversion anomalies.
[0077] Application of the Invention: When a gateway experiences data packet loss or communication interruption, it logs a red error log. This invention can be configured to immediately trigger the upload when the error log is generated, and only upload log files for the relevant time period. This allows operations engineers to receive alarms and critical logs almost in real time, quickly locate and restore faults, and ensure production continuity.
[0078] Application Scenario 3: Consumer-grade smart hardware;
[0079] Scope of application: Smart gateways, network cameras, and other devices in smart homes, operating within the user's home. When devices go offline or malfunction, customer service needs to remotely retrieve and analyze device logs, but users do not have the ability to manually extract logs.
[0080] Application of the invention: Users can trigger the "log reporting" function on the app. This action, as a specific event, triggers the device to automatically and securely upload all log files from the recent period to the server. After uploading, the device's local logs are cleaned up, protecting user privacy and providing technical support personnel with complete diagnostic data.
[0081] This application, during device runtime, utilizes the integrated EasyLogger component to output logs. Based on preset rules, different levels of output logs are color-coded to obtain colored logs. These colored logs are then continuously stored in local files according to a preset strategy, with a unique hash value generated for each. Preset trigger conditions are continuously monitored; when these conditions are met, an SFTP transfer client is launched to upload the colored logs. Once the upload is complete, a remote server returns a confirmation of receipt, and the integrity of the uploaded file is verified based on the hash value of the colored log. After the integrity verification passes, the corresponding log file successfully uploaded to the device is automatically deleted, freeing up local storage space. This application maximizes savings in device bandwidth and storage space, making it suitable for cost- and resource-sensitive IoT scenarios and addressing the pain points of resource-constrained embedded devices in log management.
[0082] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. An EasyLogger-based log management and SFTP automatic synchronization method, characterized in that, The application comprises the following steps: During the operation of the device, the application outputs logs through the integrated EasyLogger component, adds color identification to the output logs of different levels based on preset rules, and obtains colored logs; The colored logs are stored in local files based on preset strategies, and a unique hash value is generated for the colored logs stored in the local files; The preset trigger conditions are continuously monitored, and when the trigger conditions are met, the SFTP transmission client is started to realize the uploading of the colored logs; After the uploading of the colored logs is completed, the remote server returns a receiving confirmation, and the integrity of the uploaded file is checked based on the hash value of the colored logs; After the integrity check of the uploaded file is passed, the corresponding log file that has been successfully uploaded on the device is automatically deleted to release the local storage space.
2. The method of claim 1, wherein the EasyLogger-based log management and SFTP automatic synchronization method is characterized by, Color identification is added to the output log information of different levels based on preset rules, which simultaneously acts on the terminal display and the writing of the local log file.
3. The method of claim 1, wherein the EasyLogger-based log management and SFTP automatic synchronization method is characterized by, The hash value is saved together with the colored logs as the identity of the colored logs stored in the local files.
4. The method for EasyLogger-based log management and SFTP automatic synchronization according to claim 3, characterized in that, Each hash value serves as an upload check identifier.
5. The method for EasyLogger-based log management and SFTP automatic synchronization according to claim 1, characterized in that, The upload level is set according to different color identifications.
6. The method of claim 1, wherein the method is based on EasyLogger-based log management and SFTP automatic synchronization. The preset trigger conditions are continuously monitored, and when the trigger conditions are met, the SFTP transmission client is started to realize the uploading of the colored logs, including the following steps: A secure connection is established with the remote SFTP server through the integrated libssh library, and the colored log file to be uploaded is transmitted to the specified directory of the server.
7. The method of claim 1, wherein the method is based on EasyLogger-based log management and SFTP automatic synchronization. The colored logs are compressed before being uploaded.
8. The method of claim 1, wherein the EasyLogger-based log management and SFTP automatic synchronization method is characterized by, When the colored logs are uploaded, the colored logs that meet the preset threshold are uploaded based on the configured upload level and the preset threshold, and the colored logs that do not meet the threshold are filtered.
9. The method of claim 1, wherein the method is based on EasyLogger-based log management and SFTP automatic synchronization, and When the signal is weak or the traffic is expensive, the uploading can be delayed or only the key logs can be cached.
10. The method of claim 1, wherein the method is based on EasyLogger-based log management and SFTP automatic synchronization. When the integrity check of the uploaded file fails, the re-uploading mechanism is triggered.