Method for collecting logs based on satellite communication terminals

Abstract

The application provides a method based on satellite communication terminal log collection, which is used for coping with various sudden situations, accurately saving terminal system log and state reporting, providing data support for future troubleshooting, and restoring scene. The method comprises a local cache mechanism and a log transmission mechanism. The local cache mechanism is a combination of memory cache and disk cache. The memory cache is used for storing the latest log, and the disk cache is used for persistently storing system log and state under abnormal conditions. The log transmission mechanism avoids log loss and better realizes abnormal scene positioning through the ways of compression return, configurable upload, and clearing terminal log after successful upload.

Description

Technical Field

[0001] This invention belongs to the field of high-throughput satellite communication technology, and specifically relates to a method for log collection. Background Technology

[0002] Both satellite terminals and traditional network terminals face challenges such as unstable links, harsh terminal environments, and high technical complexity and equipment requirements. During this process, various unexpected situations may arise. Log processing can accurately save terminal system logs and status reports, providing data support for future troubleshooting and problem-solving, and for reproducing scenarios.

[0003] Traditional satellite terminal logs are mostly single-file records, which have the following main shortcomings:

[0004] 1. Incomplete log configuration, including log file size, log return target address and port. When the satellite ground station system is upgraded, it will be unable to notify the satellite terminal to change the log configuration in a timely manner.

[0005] 2. The log writing mechanism is incomplete, resulting in the loss or overwriting of old logs. There is no log caching or return mechanism.

[0006] 3. Log loss may occur due to power outages or system malfunctions;

[0007] 4. Log writing is coupled with system applications, and file reading and writing consume main process resources. If file reading and writing fails, the main process will be restarted.

[0008] 5. The log entries are not detailed enough. Satellite terminals need to record data related to satellite signal transmission and reception. Traditional logs only record system information and pay little attention to overall link and environment information (such as host memory, CPU usage, current latitude and longitude, signal-to-noise ratio).

[0009] 6. The logs are only passively written by the system's main process, without any active collection process;

[0010] 7. The logging lacks flexibility, and the high frequency of writes by the main process will cause frequent file read and write operations, without any threshold mechanism to control it. Summary of the Invention

[0011] To address the aforementioned problems, this invention proposes a logging method for high-throughput satellite communications. The specific technical solution is as follows:

[0012] Methods for collecting logs from satellite communication terminals include local caching mechanisms and log transmission mechanisms.

[0013] The local caching mechanism combines memory caching and disk caching. The memory cache is used to store the latest logs, while the disk cache is used to persistently store logs that failed to be returned. The memory cache is overwritten cyclically and is not affected by whether the remote log return is successful or not. The disk cache is only activated when the remote log return fails. When the maximum storage space allocated by the log server is exceeded, the disk cache will no longer store logs.

[0014] The log transmission mechanism is as follows: the satellite terminal uploads local log files through a queue. After the previous log request is successfully sent, the next log file is reported. Specifically, after obtaining and compressing the log data, the satellite terminal sends a return request to the log server. After the log server successfully decompresses and saves the log data, the satellite terminal deletes the corresponding log file, the current log return process ends, and the satellite terminal continues to return the next log file.

[0015] Each time a log is transmitted back, data is read from memory first. If the memory data upload fails, a disk data queue is not created. If the memory data transmission succeeds, a disk data queue is created for uploading.

[0016] The log processing process handles log file read and write operations independently, and communicates with the main process via UDP.

[0017] After receiving a UDP message from the main process, the log processing process checks the time difference between the two received UDP messages. If it is less than a set threshold, the main process is considered to be outputting logs too quickly, and the logs are cached in the memory of the log processing process itself. Every time the threshold time expires, the log processing process writes the logs to the memory cache area in the local caching mechanism once. If the time difference is greater than the set threshold, the main process is considered to be outputting logs at a moderate pace, and the log processing process receives each piece of data and writes it to the memory cache area in the local caching mechanism.

[0018] After the terminal goes online, the log configuration parameters of the terminal are dynamically configured in the satellite ground station network management system, including the address and port of the log server for log transmission, the size or proportion of local disk storage space occupied by the satellite terminal logs, and the number of concurrent log uploads.

[0019] Log collection not only provides log information generated by the system's main process, but also includes beam information, forward frequency, antenna status, terminal registration status, signal-to-noise ratio, current time, current latitude and longitude, current transmit power, terminal version, serial number, power-on duration, and corresponding firmware and software version information. It can also proactively collect system upgrade status, terminal registration status, ACU connection status, and beam status information.

[0020] Beneficial effects

[0021] 1. Ensure terminal logs are not lost and are strictly verified;

[0022] 2. Ensure that terminal information can be traced back historically;

[0023] 3. Ensure that satellite terminal logs can be modified and transmitted back in a timely manner when the central station server changes;

[0024] 4. Reduce log read / write anomalies and system resource consumption. Attached Figure Description

[0025] Figure 1 1. Principle block diagram of the present invention.

[0026] Figure 2 Flowchart of this invention. Detailed Implementation

[0027] The following combination Figure 1 Describe the log collection process.

[0028] 1. Log configuration parameters are sent to the satellite terminal via the log server on the right and received by the log process;

[0029] 2. The log process receives log information generated by the system process, i.e. the system main process, via UDP messages, and collects necessary local information, including beam information, forward frequency, antenna status, terminal registration status, signal-to-noise ratio, current time, current latitude and longitude, current transmit power, terminal version, serial number, power-on duration, and corresponding firmware and software version information. It can also actively collect system upgrade status, terminal registration status, ACU connection status, and beam status information.

[0030] 3. While storing logs in memory, the log process attempts to upload logs generated over a period of time (e.g., 5 minutes). If the upload fails, the logs are stored on the local disk for persistent storage. If the storage space exceeds the maximum value allocated by the log server, persistent storage will not be performed until the persistent file is successfully uploaded and the local log file is deleted to free up space.

[0031] 4. The upload process uses gzip compression to reduce bandwidth usage. The satellite ground station, i.e., the log server, decompresses the received log information. If decompression is successful, it returns a success message to the satellite terminal, which then deletes the locally cached log files.

[0032] The following describes in detail how the present invention solves the various problems mentioned in the background art.

[0033] 1. Regarding the aforementioned issue 1, after the terminal goes online, the satellite ground station network management system can configure the log server address and port for future log transmissions from the terminal, and set a size limit for the satellite terminal log cache directory to prevent the log cache from occupying too much disk space.

[0034] To address issues 2 and 3 mentioned above, this method employs a mechanism combining local memory caching and disk caching of logs, along with intermittent log back to the server. First, after logs are generated, they are written to memory. Since memory read / write speeds are faster than physical disks, it is more conducive to log storage. Typically, each log file is a maximum of 5MB, and memory can store a maximum of 5 files. When the 5th file is full, the first file is overwritten and written to the log. Simultaneously with log generation, if the log server address and port set in point 1 exist, a log back is initiated to the log server. The success of the remote back is independent of whether the log is written to local memory. However, if the remote log back fails, the log content is stored locally as a compressed log file on the disk. The advantages of combining local memory caching and disk caching for logs are as follows: Memory caching is unaffected by the success or failure of remote data transmission, always storing the latest logs through a circular overwrite mechanism; disk caching only activates when remote data transmission fails, and is used to persistently store failed transmission logs. Logs within a certain period after a transmission failure are crucial for analyzing system anomalies, reflecting poor network conditions at that time; furthermore, when the maximum storage space allocated by the log server is exceeded, the disk cache will stop storing logs, ensuring that logs are retained during critical periods while maximizing system resource conservation. It should be noted that the specific mechanism for log transmission to the satellite ground station server (i.e., the log server) is as follows: during each log transmission, data is first read from memory. If the memory data upload fails, a disk data queue is not established; if the memory data transmission succeeds, a disk data queue is established for uploading. The transmission order follows a first-in, first-out (FIFO) principle. When the returned data comes from memory, the log file content generated within a specified time period is read from local memory and compressed using gzip. When the returned data comes from disk, the previously cached log file archive is decompressed, and its content is read. After obtaining the log data, an HTTP request is sent to the log server. The log server receives the data stream from the HTTP request body, stores it on the log server after receiving all data, and then decompresses it. If decompression is successful, a success status is sent to the satellite terminal, and the uploaded file content is deleted from the satellite terminal. The log server then completes the transmission process with the satellite terminal and continues reading the received log content and storing it in the database. The satellite terminal then continues decompressing, reading, and uploading the next log file, waiting for a response. During this process, the satellite terminal uploads local log files serially or in parallel using a queue. This is mainly because uploading log files is a bandwidth-intensive process; therefore, the next log file will only be sent if the previous request is successful. Whether to upload serially or in parallel depends on the configuration issued by the log server. If the concurrency is 1, it can be considered serial. If the concurrency is greater than 1, it will be uploaded in parallel according to the concurrency. It mainly depends on the concurrency defined based on network bandwidth.

[0035] During this process, power outages and system anomalies are inevitable. Since the log caching mechanism is to persistently store the logs on the disk, log loss in the event of power outages or system anomalies can be avoided. At the same time, after power is restored and the network is reconnected, the above file reading process is still followed to transmit the logs back, which can better locate the abnormal scenario at that time.

[0036] 2. To address the issues mentioned in points 4 and 7 above, this invention designs a local and remote storage mechanism for log processing. Logs are first sent from the satellite terminal's main process to the log processing process via UDP messages. The satellite terminal's main process is not responsible for handling file read / write operations on the logs, thus decoupling file read / write operations from the main process and preventing disruption to its normal operation. Upon receiving the UDP message from the main process, the log processing process first caches the logs in its own memory. It then flexibly determines whether to directly perform file read / write operations by judging the time interval between two consecutive UDP messages (e.g., 200ms), or to cache the log information in its own memory within a specified time (e.g., 2s) and then perform the file read / write operation all at once. This is primarily because frequent file read / write operations on the local disk not only reduce disk lifespan but also increase the system resources used by the process, and can also lead to process blocking issues caused by unsuccessful file read / write operations.

[0037] 3. Regarding issues 5 and 6 mentioned above, log collection not only provides log information generated by the system's main process, but also includes beam information, forward frequency, antenna status, terminal registration status, signal-to-noise ratio, current time, current latitude and longitude, current transmit power, terminal version, serial number, power-on duration, and corresponding firmware and software version information. It can also proactively collect system upgrade status, terminal registration status, ACU connection status, and beam status information.

Claims

1. A method for collecting logs based on a satellite communication terminal, characterized in that: The local cache mechanism and the log transmission mechanism are included. The local cache mechanism is a combination of memory cache and disk cache, the memory cache is used for storing the latest logs, and the disk cache is used for persistent storage of the failed log back transmission; The memory cache is cyclically overwritten and is not affected by the success of the log back transmission; the disk cache is started when the remote log back transmission is unsuccessful and is stopped when the maximum storage space is exceeded; The log transmission mechanism is that the satellite terminal uploads the local log file through a queue mode, specifically, after obtaining and compressing the log data, the satellite terminal initiates a back transmission request to the log server, after the log server successfully decompresses and saves the log data, the satellite terminal deletes the corresponding log file, and the current log back transmission process is ended, and the satellite terminal station continues the back transmission of the next log.

2. The method for collecting logs based on a satellite communication terminal according to claim 1, characterized in that: When the log back transmission is performed each time, data in the memory is preferentially read, if the memory data uploading is unsuccessful, the queue of the disk data is not established, and if the memory data back transmission is successful, the queue of the disk data is established for uploading.

3. The method for collecting logs based on a satellite communication terminal according to claim 1 or 2, characterized in that: The log processing process independently processes the read and write operations of the log file, and the log processing process and the main process communicate through udp.

4. The method for collecting logs based on a satellite communication terminal according to claim 3, characterized in that: After receiving the udp message, the log processing process judges the time difference between the two received udp messages, if the time difference is less than a set threshold value, it is considered that the main process outputs the log too fast, and the log is buffered in the memory of the log processing process itself, every time the threshold value is exceeded, the log processing process writes the log into the memory cache area in the local cache mechanism at one time, if the time difference is greater than the set threshold value, it is considered that the main process outputs the log moderately, and the log processing process receives each data and writes it into the memory cache area in the local cache mechanism.

5. The method for collecting logs based on a satellite communication terminal according to claim 1 or 2 or 4, characterized in that: After the terminal is online, the log configuration parameters of the terminal are dynamically configured in the satellite ground station network management system, including the log server address and port of the log back transmission, the satellite terminal log local disk storage space occupation size or proportion, and the log uploading concurrency.

6. The method for collecting logs based on a satellite communication terminal according to claim 5, characterized in that: The log collection not only provides the log information generated by the system main process, but also includes beam information, forward frequency, antenna state, terminal registration state, signal-to-noise ratio, current time, current latitude and longitude, current transmission power, terminal version, serial number, boot-up duration, corresponding firmware and software version information, and can also actively collect system upgrade state, terminal registration state, acu connection state, and beam state information.

Images