Multi-tool remote log gathering and copying method and system based on disk adaptation

Abstract

The invention discloses a multi-tool remote log gathering and copying method and system based on disk adaptation, and belongs to the technical field of log management. The method comprises the following steps: in a remote command sending and receiving stage, a control end generates a specific log copying command for different tools according to a preset rule of log storage positions of the tools, and sends the specific log copying command to a corresponding test end host through a network protocol; in the log copying and transmitting stage, according to the analysis result, an obtaining mode is selected to obtain a log, log copying operation is executed, then the test end host carries out data encryption on a log file, and then the log file is sent back to the control end through a network transmission protocol; and a log processing and downloading stage: after receiving the log files from each test end host, the control end automatically classifies and arranges the log files according to a preset classification rule, and provides a log downloading interface for a user. For different types of tools, the system automatically matches the optimal acquisition mode, and compared with manual judgment and operation, the efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of log management technology, and in particular to a multi-tool remote log aggregation and copying method and system based on disk adaptation. Background Art

[0002] In the information technology field, efficient collection and analysis of logs generated by various tools is crucial. However, current log acquisition faces many challenges: The acquisition methods are complex and diverse: different tools generate logs in different ways, with some using specific operating system commands, some relying on the tool's own interface, and others requiring acquisition from databases or message queues. There is a lack of a unified and efficient acquisition mechanism. Existing technologies have the following problems: 1) Dispersed log storage: Logs may be stored in different paths or even different storage media, making them difficult to find and collect.

[0003] 2) Lack of automation and standardization: Currently, log collection mostly relies on manual labor, which is inefficient and error-prone. In addition, the acquired logs lack standardized naming and organization, which is not conducive to subsequent analysis and use.

[0004] 3) Transmission security: During the transmission of logs from the acquisition end to the management end, data is vulnerable to theft or tampering, posing a security risk. With the increasing sophistication of cyberattacks, ensuring the security of logs during transmission has become a critical issue. Some existing transmission methods do not encrypt log data, making it vulnerable to theft or tampering during network transmission. This can lead to the leakage of sensitive information and threaten system security.

[0005] 4) Complex and inconsistent log acquisition methods: Different tool developers use different logging and acquisition methods. For example, some system-level tools rely on operating system-specific commands to acquire logs, such as the Windows Event Viewer command and the Linux journalctl command. Meanwhile, some applications provide custom interfaces for log acquisition. This diversity requires operations personnel to familiarize themselves with multiple acquisition methods, increasing learning costs and operational complexity.

[0006] 5) Manual operations are inefficient and error-prone: Manually searching and collecting log files is a time-consuming and labor-intensive task. Operations and maintenance personnel must rely on memory and experience to locate log files across numerous servers and storage devices, which can easily lead to missing important information. Furthermore, manual operations are prone to errors during the copying and transfer process, such as file corruption and data loss, impacting log integrity and availability.

[0007] 6) Log management is chaotic: Logs often lack unified naming and classification rules, making it difficult to quickly locate and retrieve required information. For example, logs from different tools may use different naming methods, some by date, others by functional module. This lack of standardized management reduces the log's usefulness. Summary of the Invention

[0008] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a multi-tool remote log aggregation and copying method and system based on disk adaptation.

[0009] The object of the present invention is achieved through the following technical solutions: In a first aspect, the present invention provides a multi-tool remote log aggregation and copying method based on disk adaptation, comprising the following steps: During the remote command sending and receiving phase, the control end automatically discovers test hosts with tools installed on the local area network or within a specified range through network scanning. Based on the preset rules for log storage locations for different tools, the control end generates specific log copy commands and sends them to the corresponding test hosts via the network protocol. During the log copy and transmission phase, after receiving the log copy command, the test host parses the command content, confirms the path of the log file to be copied, and then selects an acquisition method based on the parsed result to obtain the log. It then performs the log copy operation and copies the specified log file to a temporary storage location. The test host then encrypts the temporarily stored log file and sends it back to the control end via the network transmission protocol. During the log processing and downloading phase, after the control end receives the log files from each test end host, it automatically classifies and organizes them according to the preset classification rules and provides the user with a log download interface.

[0010] Preferably, the remote command sending and receiving stage further includes the following steps: Initialize the control end system resources and load configuration information; The control end initiates a TCP access probe to the test end tool based on the IP address in the added host information, sends a TCP connection request through a specific port, and attempts to establish a connection with the test end tool to determine whether the tool is started. If the TCP access probe finds that the tool is not started, the control end sends a specific command, confirms its operating system type based on the response information returned by the test end host, and then queries the system specific location based on the operating system type to determine whether the tool has been installed. If it is detected that the tool is not installed, the control end sends an installation command based on the operating system type. If it is found that the tool is installed but not started, the control end sends a startup command and calls the startup instruction according to the operating system type to start the tool. The control end also records the identified IP address of the test end host, the operating system type, and the tool installation and startup status. After the tool is started, it determines the specific method for obtaining logs based on the characteristics of different tools and the preset log acquisition strategy, and generates a specific log copy command; Create a TCP socket through the network programming interface provided by the QT development framework, establish a TCP connection based on the IP address of the test host and the specified port number, and after a successful connection, serialize the log copy command and send it to the test end through the TCP socket.

[0011] Preferably, the log copying and transmission stage further includes the following steps: The test host starts the network monitoring service and configures the client parameters; Use the network communication module based on the QT development framework to receive the log copy command and save it; Read the log copy command, parse the command format according to the preset rules, extract the acquisition method and related parameter information, and obtain the log according to the acquisition method; Get the current time, format it, and determine the host type by reading the system configuration file or environment variable. Then name the log file according to the naming rules. Then select a compression format to compress the log file to obtain a log compression package. According to the control terminal configuration, select the network transmission protocol, configure the protocol parameters, encrypt the log compression package and transmit it back to the control terminal.

[0012] Preferably, the log processing and downloading stage further includes the following steps: The control end starts network monitoring, configures server-side parameters according to the selected network transmission protocol, and uses the protocol library to receive and save log compression packages; Determine the compression format of the log compression package, and use the corresponding decompression library to decompress the log files in the log compression package to the specified directory; Read the preset classification rules, determine the classification dimensions, traverse the decompressed log files, and move the log files to the corresponding folders according to the log information in the log files and the classification rules; A download button is provided for each log file. After the user clicks the download button, the program calls the underlying download function module, obtains the storage path of the selected log file, starts the download process, and provides a breakpoint resume function. When the download is interrupted, the downloaded part is recorded and the download starts from the breakpoint when downloading again.

[0013] Preferably, data encryption is performed using SSL / TLS encryption protocol.

[0014] Preferably, the acquisition method includes Windows / Linux special command acquisition, specified path acquisition, specified tool interface acquisition, database query acquisition, message queue subscription acquisition and script automation acquisition.

[0015] A second aspect of the present invention provides: a multi-tool remote log aggregation and copying system based on disk adaptation, for implementing any of the above-mentioned multi-tool remote log aggregation and copying methods based on disk adaptation, comprising: The remote command sending and receiving module is used to use the control terminal to automatically discover the test host with the tool installed on the local area network or within a specified range through network scanning. The control terminal generates specific log copy commands for different tools according to the preset rules of their log storage locations and sends them to the corresponding test host through the network protocol. The log copy and transmission module is used to parse the command content after the test host receives the log copy command, confirm the log file path to be copied, and then select the acquisition method based on the parsing result to obtain the log, perform the log copy operation, and copy the specified log file to a temporary storage location. The test host then encrypts the temporarily stored log file and sends it back to the control end through the network transmission protocol. The log processing and download module is used to automatically classify and organize the log files received from each test host on the control end according to the preset classification rules, and provide the user with a log download interface.

[0016] The third aspect of the present invention provides: a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by a processor, any of the above-mentioned multi-tool remote log aggregation and copying methods based on disk adaptation is implemented.

[0017] A fourth aspect of the present invention provides: a computer program product comprising instructions, which, when executed on a terminal, enables the terminal to execute any one of the multi-tool remote log aggregation and copying methods based on disk adaptation.

[0018] The beneficial effects of the present invention are: 1) Improved acquisition efficiency: Multiple log acquisition methods are integrated, eliminating the need to manually try different methods one by one, saving time. For example, the system automatically matches the optimal acquisition method for different tool types, significantly improving efficiency compared to manual judgment and operation. The automated acquisition process reduces manual intervention. From host scanning to log acquisition, the entire process is completed automatically, eliminating the tedious manual search and collection process and improving acquisition speed.

[0019] 2) Enhanced data integrity and accuracy: Automated operations reduce human errors, such as file copying errors and data omissions, ensuring the completeness and accuracy of acquired logs. Standardized naming and categorization facilitate rapid log location and retrieval, increasing log value and ensuring accurate and reliable analysis results.

[0020] 3) Improved transmission security: SSL / TLS encryption technology is used to prevent log theft or tampering during transmission, protecting sensitive information. This effectively avoids data leakage risks and ensures system security in complex network environments.

[0021] 4) Reduced Operation and Maintenance Costs: Reduced manual operations and labor costs. Operation and maintenance personnel no longer need to spend extensive time and energy on log acquisition and organization, allowing them to devote more resources to system optimization and troubleshooting. Unified management mechanisms and standardized operating procedures reduce learning costs, enabling new operators to quickly get started and improve overall operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is the overall flow chart of the multi-tool remote log aggregation and copy method based on disk adaptation; Figure 2 This is the control end flow chart; Figure 3 This is the test end flow chart. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0024] First, the definitions of some terms of the present invention: 1) IP address range scanning: A network scanning technology that traverses a specified IP address range and detects whether the hosts in the range are alive and open ports, etc., to discover devices on the network.

[0025] 2) Service Discovery Protocol: A protocol that allows devices to automatically discover each other and the services they provide on the network. It can help quickly locate hosts on the network that have specific tools installed.

[0026] 3) SSH (Secure Shell): A protocol used to provide secure remote login and other network services over an insecure network, ensuring data security through encrypted communications.

[0027] 4) SMB (Server Message Block): A network file sharing protocol that can be used to share files, printers, and other resources between different operating systems, enabling command transmission and file interaction between the control end and the test end.

[0028] 5) FTP (File Transfer Protocol): A standard protocol for transferring files over the network, enabling the transfer of log files from the test end to the control end.

[0029] 6) SFTP (SSH File Transfer Protocol): A file transfer protocol based on the SSH protocol that provides encrypted file transfer capabilities to ensure secure log transmission.

[0030] 7) SSL / TLS (Secure Sockets Layer / Transport Layer Security): A protocol used to encrypt data in network communications, preventing data from being stolen or tampered with during transmission and ensuring the security of log data transmission.

[0031] See Figure 1-Figure 3 The first aspect of the present invention provides: a multi-tool remote log aggregation and copying method based on disk adaptation, comprising the following steps: During the remote command sending and receiving phase, the control end automatically discovers test hosts with tools installed on the local area network or within a specified range through network scanning. Based on the preset rules for log storage locations for different tools, the control end generates specific log copy commands and sends them to the corresponding test hosts via the network protocol. During the log copy and transmission phase, after receiving the log copy command, the test host parses the command content, confirms the path of the log file to be copied, and then selects an acquisition method based on the parsed result to obtain the log. It then performs the log copy operation and copies the specified log file to a temporary storage location. The test host then encrypts the temporarily stored log file and sends it back to the control end via the network transmission protocol. During the log processing and downloading phase, after the control end receives the log files from each test end host, it automatically classifies and organizes them according to the preset classification rules and provides the user with a log download interface.

[0032] In this embodiment, the control side manages and coordinates the entire log acquisition process, including identifying the test side host, issuing log copy commands, and receiving and processing logs transmitted from the test side. The test side, deployed on each host with the tool installed, is responsible for receiving commands from the control side, executing log copy operations, and transmitting logs back to the control side.

[0033] Intelligent remote command sending and receiving mechanism allows the control end to identify test hosts: The control end uses network scanning technology to automatically discover test hosts with relevant tools installed within the local area network or a specified range. It can quickly and accurately locate each test host using methods such as IP address segment scanning and service discovery protocols. Sending specific log copy commands: The control end generates specific log copy commands for different tools based on preset log storage locations. These commands can precisely locate the tool's log files in different disk locations. For example, for Tool A, whose storage location is known, the command can be precisely specified to a specific file path such as "C:\Program Files\ToolA\Logs\Current.log." Commands are sent to the corresponding test host via network protocols such as SSH and SMB.

[0034] Test-side log copying and transmission, receiving and executing commands: After receiving the log copy command sent by the control side, the test-side host parses the command content and determines the path of the log file to be copied. Based on the parsing results, the test-side host then retrieves the log file through various methods, including Windows / Linux special commands, specified paths, specified tool interfaces, database queries, message queue subscriptions, and script automation. The test-side host then executes the log copy operation, copying the specified log file to a temporary storage location. Remote log transmission: The test-side host sends the temporarily stored log file back to the control side host via a network transfer protocol (such as FTP or SFTP). During the transmission process, data encryption technologies (such as SSL / TLS) can be used to ensure the security of the log data and prevent data theft or tampering during transmission.

[0035] Control-side log processing and downloading, with automated classification and organization: After the control-side host receives log files from each test end, it automatically classifies and organizes them according to preset classification rules. For example, they can be classified by tool type, date, log level, and other dimensions. Log files generated by the same tool at different times are organized into the same folder to facilitate user search and management. Provide a download interface: The control side provides users with a convenient log download interface. Users can download the required log files to their local computer with simple operations through the web interface, command line tools, and other methods. During the download process, the system can provide progress display, breakpoint resumption, and other functions to enhance the user experience.

[0036] In some embodiments, the remote command sending and receiving stage further includes the following steps: Initialize the control end system resources and load configuration information; The control end initiates a TCP access probe to the test end tool based on the IP address in the added host information, sends a TCP connection request through a specific port, and attempts to establish a connection with the test end tool to determine whether the tool is started. If the TCP access probe finds that the tool is not started, the control end sends a specific command, confirms its operating system type based on the response information returned by the test end host, and then queries the system specific location based on the operating system type to determine whether the tool has been installed. If it is detected that the tool is not installed, the control end sends an installation command based on the operating system type. If it is found that the tool is installed but not started, the control end sends a startup command and calls the startup instruction according to the operating system type to start the tool. The control end also records the identified IP address of the test end host, the operating system type, and the tool installation and startup status. After the tool is started, it determines the specific method for obtaining logs based on the characteristics of different tools and the preset log acquisition strategy, and generates a specific log copy command; Create a TCP socket through the network programming interface provided by the QT development framework, establish a TCP connection based on the IP address of the test host and the specified port number, and after a successful connection, serialize the log copy command and send it to the test end through the TCP socket.

[0037] In this embodiment, the process steps of the control end are as follows: 1) Control end startup: Initialize the control end system resources, load configuration information, and prepare for log acquisition and management.

[0038] 2) The control side performs a network scan to identify the test side host: a) TCP access detection: Based on the IP address in the added host information, the control side initiates a TCP access detection for the test side tool. By sending a TCP connection request to a specific port, it attempts to establish a connection with the test side tool to determine whether the tool is running. b) System Identification and Tool Detection: If the TCP access detection indicates that the tool is not running, the control side sends a specific command and, based on the response information returned by the test side, determines the operating system type (Windows or Linux). After confirming the system type, it queries specific system locations (Windows systems query the registry or specific installation directories; Linux systems check the package management system installation records) to determine whether the relevant tool is installed. c) Tool Installation and Startup: If the relevant tool is not installed, the control side sends an installation command based on the operating system type. For Windows systems, this involves invoking a PowerShell script or a specific installer command for installation. For Linux systems, this involves invoking an installation command based on the package management tool used (such as apt-get or yum). If the tool is installed but not running, the control side sends a startup command, invoking the appropriate startup instructions for the operating system type to start the tool. d) Information logging: Records detailed information about the identified test host, including IP address, operating system type, tool installation and startup status, etc., to provide a basis for subsequent sending of log acquisition instructions.

[0039] 3) Determine the log acquisition method and generate instructions: Based on the characteristics of different tools and the preset log acquisition strategy, determine the specific method for acquiring logs and generate corresponding detailed instructions. a) Windows system-specific command acquisition: If the tool is based on the Windows system and supports specific commands for acquiring logs, such as acquiring system logs, generate the command "wevtutilqeSystem / rd:true / f:text>C:\Logs\SystemLog.txt" (assuming this command is used to acquire Windows system logs and save them to a specified path). b) Specified tool interface acquisition: If logs are acquired through a specified tool interface, such as a tool that provides a Python API for acquiring logs, generate instructions that include information such as the interface call parameters, such as "import my_tool_api; my_tool_api.get_logs (' / tmp / tool_logs.txt')" (assuming this code is used to acquire logs through the tool's Python API and save them to a specified file). c) Database Query: To obtain logs via database query, determine the database type (e.g., MySQL, Oracle), connection information such as the host address, port, username, and password, and the query statement, such as "SELECT * FROM logs_table WHERE log_type = 'error'" (assuming this statement is used to query error-type logs from a table named logs_table). d) Other Acquisition Methods: Other acquisition methods are similar to the above steps, generating the corresponding detailed instructions.

[0040] 4) The control end sends commands to the test end via a network protocol. Regardless of whether the test host's operating system is Linux or Windows, command transmission is based on the TCP protocol. Leveraging QT's cross-platform nature, a TCP socket is created using the network programming interface provided by QT. A TCP connection is established based on the test host's IP address and the specified port number. Once the connection is established, a command containing log acquisition methods and detailed instructions is serialized and sent to the test end via the TCP socket.

[0041] In some embodiments, the log copying and transmission phase further includes the following steps: The test host starts the network monitoring service and configures the client parameters; Use the network communication module based on the QT development framework to receive the log copy command and save it; Read the log copy command, parse the command format according to the preset rules, extract the acquisition method and related parameter information, and obtain the log according to the acquisition method; Get the current time, format it, and determine the host type by reading the system configuration file or environment variable. Then name the log file according to the naming rules. Then select a compression format to compress the log file to obtain a log compression package. According to the control terminal configuration, select the network transmission protocol, configure the protocol parameters, encrypt the log compression package and transmit it back to the control terminal.

[0042] In this embodiment, the tester's process steps are as follows: 1) Start the tester and listen: The tester starts the network listening service, configures client parameters based on the TCP protocol, and waits for commands from the control terminal using QT's cross-platform features. This step initializes the basic environment for communication between the tester and the control terminal, ensuring that the tester can receive commands from the control terminal.

[0043] 2) Receive and save commands: Use the QT-based network communication module to receive commands from the control terminal, including log acquisition methods and detailed instructions, and save them to a designated location. This ensures accurate command reception and storage, preparing for subsequent parsing and execution.

[0044] 3) Command Parsing and Operation Determination: Read the received command file contents and extract the log acquisition method and related parameter information according to the preset command format parsing rules. Determine the log acquisition method, including whether to acquire logs through Windows / Linux special commands, designated paths, designated tool interfaces, database queries, message queue subscriptions, or script automation.

[0045] 4) Obtain logs according to the selected method.

[0046] 5) Log Naming and Compression: a) Naming Information Acquisition: Obtain the current time and format it in YYYYMMDDHHMMSS format. Determine the host type by reading system configuration files or environment variables. b) Log Naming: Follow the naming rules and name the log file something like "20240101120000_dev_host_log.txt." c) Compression: Determine the compression format. If the zip format is selected, use a compression library (such as the zipfile library) to compress the log file into a compressed package named "20240101120000_dev_host_log.zip." If the tar.gz format is selected, use the tarfile library for compression.

[0047] 6) Transmitting the compressed log package: Based on the control-side configuration, select the appropriate network transmission protocol (such as FTP or SFTP) and configure the relevant transmission protocol parameters, such as the FTP server address, port number, username, and password. Use the corresponding protocol library (such as ftplib for FTP or the Paramiko SFTP client for SFTP) to transmit the compressed log package back to the control-side host. During the transmission process, use data encryption technology (such as SSL / TLS) to ensure the security of the log data.

[0048] In some embodiments, the log processing and downloading stage further includes the following steps: The control end starts network monitoring, configures server-side parameters according to the selected network transmission protocol, and uses the protocol library to receive and save log compression packages; Determine the compression format of the log compression package, and use the corresponding decompression library to decompress the log files in the log compression package to the specified directory; Read the preset classification rules, determine the classification dimensions, traverse the decompressed log files, and move the log files to the corresponding folders according to the log information in the log files and the classification rules; A download button is provided for each log file. After the user clicks the download button, the program calls the underlying download function module, obtains the storage path of the selected log file, starts the download process, and provides a breakpoint resume function. When the download is interrupted, the downloaded part is recorded and the download starts from the breakpoint when downloading again.

[0049] In some embodiments, data encryption is performed using the SSL / TLS encryption protocol.

[0050] In some embodiments, the acquisition method includes Windows / Linux special command acquisition, specified path acquisition, specified tool interface acquisition, database query acquisition, message queue subscription acquisition, and script automation acquisition.

[0051] In this embodiment, the specific implementation steps for each acquisition method are as follows: a) Windows / Linux special command acquisition: If logs are determined to be acquired using Windows or Linux special commands, the operating system type is first determined. If the operating system is Windows, a Windows command line tool is called (e.g., using Python's subprocess module to call commands) to execute the corresponding command, such as "wevtutil qeSystem / rd:true / f:text > C:\Logs\SystemLog.txt," and the command execution result is saved as a log file. If the operating system is Linux, a similar Linux command is called, such as "journalctl -u my_service > / tmp / my_service_log.txt," to acquire the corresponding log information and save it as a log file.

[0052] b) Specified path acquisition: Extract the specified path information from the command. Determine the operating system type. If it's a Windows system, use file operation functions (such as those in the Python os module), such as "xcopy C:\Program Files\ToolX\Logs*.* D:\Temp\Logs\ / E / Y" to copy the log files in the specified path to a temporary storage location. If it's a Linux system, use the "cp -r / path / to / source_logs / tmp / temp_logs" command to copy the log files.

[0053] c) Obtaining a specified tool interface: Extract the interface information and parameters from the command. Based on the interface type provided by the tool, load the corresponding interface library (for example, if the tool provides a Python API, import the relevant module). Call the interface function or method, such as "import my_tool_api; my_tool_api.get_logs (' / tmp / tool_logs.txt')", to obtain the log and save it to the specified file.

[0054] d) Database query acquisition: Extract database connection information (such as database type, host address, port, username, password, etc.) and query statements from the command. Based on the database type, import the corresponding database connection library (for example, the mysql.connector library for MySQL). Use the connection information to establish a connection to the database, execute the query statement, and obtain relevant log data. Query results are organized and saved as a log file, such as " / tmp / database_log.txt".

[0055] e) Message Queue Subscription and Retrieval: Extract the message queue information from the command (such as the message queue server address, port, queue name, and authentication information). Depending on the message queue type, import the corresponding client library (for example, the pika library is used for RabbitMQ message queues). Use the relevant information to connect to the message queue server, declare the queue, and set a callback function to process received messages. In the callback function, organize the received log messages and save them to a log file, such as " / tmp / queue_log.txt".

[0056] f) Automated script acquisition: Extract the script path and execution parameters from the command. Determine the script language type. If it's a Python script, use the script interpreter (e.g., the python command) to execute the script, for example, "python / path / to / your_script.py arg1 arg2." The script follows pre-set logic, which may involve system command calls, file operations, data processing, and other steps. It then acquires and organizes log information, ultimately generating a log file.

[0057] A second aspect of the present invention provides: a multi-tool remote log aggregation and copying system based on disk adaptation, for implementing any of the above-mentioned multi-tool remote log aggregation and copying methods based on disk adaptation, comprising: The remote command sending and receiving module is used to use the control terminal to automatically discover the test host with the tool installed on the local area network or within a specified range through network scanning. The control terminal generates specific log copy commands for different tools according to the preset rules of their log storage locations and sends them to the corresponding test host through the network protocol. The log copy and transmission module is used to parse the command content after the test host receives the log copy command, confirm the log file path to be copied, and then select the acquisition method based on the parsing result to obtain the log, perform the log copy operation, and copy the specified log file to a temporary storage location. The test host then encrypts the temporarily stored log file and sends it back to the control end through the network transmission protocol. The log processing and download module is used to automatically classify and organize the log files received from each test host on the control end according to the preset classification rules, and provide the user with a log download interface.

[0058] The third aspect of the present invention provides: a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by a processor, any of the above-mentioned multi-tool remote log aggregation and copying methods based on disk adaptation is implemented.

[0059] A fourth aspect of the present invention provides: a computer program product comprising instructions, which, when executed on a terminal, enables the terminal to execute any one of the multi-tool remote log aggregation and copying methods based on disk adaptation.

[0060] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A multi-tool remote log aggregation and copying method based on disk adaptation, characterized by: The following steps are involved: During the remote command sending and receiving phase, the control end automatically discovers test hosts with tools installed on the local area network or within a specified range through network scanning. Based on the preset rules for log storage locations for different tools, the control end generates specific log copy commands and sends them to the corresponding test hosts via the network protocol. During the log copy and transmission phase, after receiving the log copy command, the test host parses the command content, confirms the path of the log file to be copied, and then selects an acquisition method based on the parsed result to obtain the log. It then performs the log copy operation and copies the specified log file to a temporary storage location. The test host then encrypts the temporarily stored log file and sends it back to the control end via the network transmission protocol. During the log processing and downloading phase, after the control end receives the log files from each test end host, it automatically classifies and organizes them according to the preset classification rules and provides the user with a log download interface.

2. The multi-tool remote log aggregation and copying method based on disk adaptation according to claim 1 is characterized in that: The remote command sending and receiving stage further includes the following steps: Initialize the control end system resources and load configuration information; The control end initiates a TCP access probe to the test end tool based on the IP address in the added host information, sends a TCP connection request through a specific port, and attempts to establish a connection with the test end tool to determine whether the tool is started. If the TCP access probe finds that the tool is not started, the control end sends a specific command, confirms its operating system type based on the response information returned by the test end host, and then queries the system specific location based on the operating system type to determine whether the tool has been installed. If it is detected that the tool is not installed, the control end sends an installation command based on the operating system type. If it is found that the tool is installed but not started, the control end sends a startup command and calls the startup instruction according to the operating system type to start the tool. The control end also records the identified IP address of the test end host, the operating system type, and the tool installation and startup status. After the tool is started, it determines the specific method for obtaining logs based on the characteristics of different tools and the preset log acquisition strategy, and generates a specific log copy command; Create a TCP socket through the network programming interface provided by the QT development framework, establish a TCP connection based on the IP address of the test host and the specified port number, and after a successful connection, serialize the log copy command and send it to the test end through the TCP socket.

3. The multi-tool remote log aggregation and copying method based on disk adaptation according to claim 1 is characterized in that: The log copying and transmission phase further includes the following steps: The test host starts the network monitoring service and configures the client parameters; Use the network communication module based on the QT development framework to receive the log copy command and save it; Read the log copy command, parse the command format according to the preset rules, extract the acquisition method and related parameter information, and obtain the log according to the acquisition method; Get the current time, format it, and determine the host type by reading the system configuration file or environment variable. Then name the log file according to the naming rules. Then select a compression format to compress the log file to obtain a log compression package. According to the control terminal configuration, select the network transmission protocol, configure the protocol parameters, encrypt the log compression package and transmit it back to the control terminal.

4. The multi-tool remote log aggregation and copying method based on disk adaptation according to claim 1 is characterized in that: The log processing and downloading stage also includes the following steps: The control end starts network monitoring, configures server-side parameters according to the selected network transmission protocol, and uses the protocol library to receive and save log compression packages; Determine the compression format of the log compression package, and use the corresponding decompression library to decompress the log files in the log compression package to the specified directory; Read the preset classification rules, determine the classification dimensions, traverse the decompressed log files, and move the log files to the corresponding folders according to the log information in the log files and the classification rules; A download button is provided for each log file. After the user clicks the download button, the program calls the underlying download function module, obtains the storage path of the selected log file, starts the download process, and provides a breakpoint resume function. When the download is interrupted, the downloaded part is recorded and the download starts from the breakpoint when downloading again.

5. The multi-tool remote log aggregation and copying method based on disk adaptation according to claim 1 is characterized in that: Use SSL / TLS encryption protocol for data encryption.

6. The multi-tool remote log aggregation and copying method based on disk adaptation according to any one of claims 1 to 5, characterized in that: The acquisition methods include Windows / Linux special command acquisition, specified path acquisition, specified tool interface acquisition, database query acquisition, message queue subscription acquisition and script automation acquisition.

7. A multi-tool remote log aggregation and copy system based on disk adaptation, characterized by: A multi-tool remote log aggregation and copying method based on disk adaptation for implementing any one of claims 1 to 6 comprises: The remote command sending and receiving module is used to use the control terminal to automatically discover the test host with the tool installed on the local area network or within a specified range through network scanning. The control terminal generates specific log copy commands for different tools according to the preset rules of their log storage locations and sends them to the corresponding test host through the network protocol. The log copy and transmission module is used to parse the command content after the test host receives the log copy command, confirm the log file path to be copied, and then select the acquisition method based on the parsing result to obtain the log, perform the log copy operation, and copy the specified log file to a temporary storage location. The test host then encrypts the temporarily stored log file and sends it back to the control end through the network transmission protocol. The log processing and download module is used to automatically classify and organize the log files received from each test host on the control end according to the preset classification rules, and provide the user with a log download interface.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are loaded and executed by the processor, the multi-tool remote log aggregation and copying method based on disk adaptation according to any one of claims 1 to 6 is implemented.

9. A computer program product comprising instructions, characterized in that: When the computer program product is run on a terminal, the terminal is enabled to execute the multi-tool remote log aggregation and copying method based on disk adaptation according to any one of claims 1 to 6.

Images