Multi-tool remote log aggregation and copy method and system based on disk adaptation

Abstract

The application discloses a kind of multi-tool remote log aggregation and copy method and system based on disk adaptation, belong to log management technical field.Method includes: remote command sending and receiving stage, control end is directed to different tool, according to the preset rule of its log storage location, generates specific log copy command, and is sent to the corresponding test end host through network protocol;Log copy and transmission stage, according to the analysis result, select acquisition mode to obtain log, execute log copy operation, then the test end host carries out data encryption to log file, then is sent back to control end through network transmission protocol;Log processing and download stage, after control end receives the log file from each test end host, according to the preset classification rule, it is automatically classified and arranged, and provides log download interface for user.The application is automatically matched with the best acquisition mode for different types of tools, compared with artificial judgment and operation, efficiency is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of log management technology, and in particular to a multi-tool remote log aggregation and copying method and system based on disk adaptation. Background Technology

[0002] In the field of information technology, efficient collection and analysis of logs generated by various tools is crucial. However, current log acquisition faces numerous challenges: acquisition methods are complex and diverse; different tools require different acquisition methods, some through specific operating system commands, some relying on the tool's own interface, and others needing to be retrieved from databases or message queues, lacking a unified and efficient acquisition mechanism. Existing technologies suffer from the following problems: 1) Dispersed log storage: Logs may be stored in different paths or even different storage media, making them difficult to find and collect.

[0003] 2) Lack of automation and standardization: Currently, logs are mostly obtained manually, which is inefficient, error-prone, and the obtained logs lack standardized naming and organization, which is not conducive to subsequent analysis and use.

[0004] 3) Transmission security issues: During the transmission of logs from the acquisition end to the management end, the data is vulnerable to theft or tampering, posing security risks. With the increasing sophistication of network attack methods, the security of logs during transmission has become a critical issue. Some existing transmission methods do not encrypt log data, making it easy for logs to be stolen or tampered with during network transmission, potentially leading to the leakage of sensitive information and threatening system security.

[0005] 4) Cumbersome and inconsistent log acquisition methods: Different tool developers use different methods for logging and acquiring logs. For example, some system-level tools rely on operating system-specific commands to acquire logs, such as the Event Viewer command in Windows and the journalctl command in Linux; while some applications provide custom interfaces to acquire logs. This diversity requires operations and maintenance personnel to be familiar with multiple acquisition methods, increasing the learning cost and operational difficulty.

[0006] 5) Manual operation is inefficient and error-prone: Manually searching and collecting log files is a time-consuming and labor-intensive task. Maintenance personnel need to rely on memory and experience to find log files among numerous servers and storage devices, easily overlooking important information. Furthermore, manual operation is prone to errors during copying and transmission, such as file corruption and data loss, affecting the integrity and availability of the logs.

[0007] 6) Disorganized log management: Acquired logs often lack unified naming and classification rules, making it difficult to quickly locate and retrieve the required information. For example, logs from different tools may use different naming methods, some by date and some by functional module, resulting in a lack of standardized management and reducing the usability of the logs. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for multi-tool remote log aggregation and copying based on disk adaptation.

[0009] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides a method for multi-tool remote log aggregation and copying based on disk adaptation, comprising the following steps: During the remote command sending and receiving phase, the control terminal automatically discovers test terminal hosts with tools installed on the local area network or within a specified range through network scanning; for different tools, the control terminal generates specific log copy commands according to the preset rules of their log storage location, and sends them to the corresponding test terminal hosts through network protocols; During the log copying and transmission phase, after receiving the log copy command, the test host parses the command content, confirms the path of the log file to be copied, and then selects the acquisition method to obtain the log based on the parsing result, executes the log copy operation, and copies the specified log file to a temporary storage location; then the test host encrypts the data in the temporarily stored log file and sends it back to the control end through the network transmission protocol; During the log processing and download phase, after receiving log files from various test hosts, the control terminal automatically categorizes and organizes them according to preset classification rules, and provides a log download interface for users.

[0010] Preferably, the remote command sending and receiving stage further includes the following steps: Initialize control system resources and load configuration information; The control terminal initiates a TCP access probe on the test tool based on the IP address in the added host information. It sends a TCP connection request through a specific port to attempt to establish a connection with the test tool, thereby determining whether the tool is running. If the TCP access probe finds that the tool is not running, the control terminal sends a specific command to confirm its operating system type based on the response information returned by the test host. Then, it queries a specific location on the system based on the operating system type to determine if the tool is installed. If the tool is not detected to be installed, the control terminal sends an installation command based on the operating system type. If the tool is found to be installed but not running, the control terminal sends a startup command to start the tool according to the operating system type. The control terminal also records the identified IP address of the test host, the operating system type, and the tool's installation and startup status. After the tool starts, it determines the specific method of obtaining logs based on the characteristics of different tools and the preset log acquisition strategy, and generates a specific log copy command. Create a TCP socket using the network programming interface provided by the QT development framework, establish a TCP connection based on the IP address and specified port number of the test host, and after successful connection, serialize the log copy command and send it to the test host through the TCP socket.

[0011] Preferably, the log copying and transmission stage further includes the following steps: Start the network listening service on the test host and configure the client parameters; The network communication module based on the QT development framework is used to receive and save log copy commands; The log copy command is read, and the acquisition method and related parameter information are extracted according to the preset command format parsing rules. The log is then retrieved based on the acquisition method. The system retrieves and formats the current time, and determines the host type by reading system configuration files or environment variables. It then names the log files according to naming rules and selects a compression format to compress the log files, resulting in a compressed log archive. Based on the control terminal configuration, select the network transmission protocol, configure the protocol parameters, encrypt the log compressed package, and then transmit it back to the control terminal.

[0012] Preferably, the log processing and download stage further includes the following steps: The control unit starts network monitoring, configures server-side parameters according to the selected network transmission protocol, and uses the protocol library to receive and save log compressed packages. Determine the compression format of the log archive and use the corresponding decompression library to perform the decompression operation, extracting the log files in the log archive to the specified directory. Read the preset classification rules, determine the classification dimensions, traverse the decompressed log files, and move the log files to the corresponding folders according to the classification rules based on the log information in the log files. A download button is provided for each log file. When the user clicks the download button, the program calls the underlying download function module, obtains the storage path of the selected log file, starts the download process, and provides a resume function. When the download is interrupted, the downloaded part is recorded, and the download will start from the breakpoint when it resumes.

[0013] Preferably, data encryption is performed using the SSL / TLS encryption protocol.

[0014] Preferably, the acquisition methods include Windows / Linux special command acquisition, acquisition via specified path, acquisition via specified tool interface, database query acquisition, message queue subscription acquisition, and script automation acquisition.

[0015] A second aspect of the present invention provides: a multi-tool remote log aggregation and copying system based on disk adaptation, used to implement any of the above-mentioned multi-tool remote log aggregation and copying methods based on disk adaptation, comprising: The remote command sending and receiving module is used by the control terminal to automatically discover test terminal hosts with tools installed on the local area network or within a specified range through network scanning; the control terminal generates specific log copy commands for different tools according to preset rules for their log storage locations, and sends them to the corresponding test terminal hosts through network protocols; The log copy and transmission module is used to parse the log copy command after the test host receives it, confirm the path of the log file to be copied, select the acquisition method to obtain the log based on the parsing result, and perform the log copy operation to copy the specified log file to a temporary storage location. Then, the test host encrypts the data in the temporarily stored log file and sends it back to the control end through the network transmission protocol. The log processing and download module is used to automatically classify and organize log files received from various test host terminals on the control terminal according to preset classification rules, and provide users with a log download interface.

[0016] A third aspect of the present invention provides: a computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, they implement any of the above-described methods for multi-tool remote log aggregation and copying based on disk adaptation.

[0017] A fourth aspect of the present invention provides: a computer program product containing instructions, which, when run on a terminal, causes the terminal to execute any of the above-described methods for multi-tool remote log aggregation and copying based on disk adaptation.

[0018] The beneficial effects of this invention are: 1) Improved acquisition efficiency: Multiple log acquisition methods are integrated, eliminating the need for manual trial and error, saving time. For example, the system automatically matches the optimal acquisition method for different types of tools, significantly improving efficiency compared to manual judgment and operation. The automated acquisition process reduces manual intervention; from scanning the host to acquiring logs, the entire process is completed automatically, avoiding the tedious process of manual searching and collection, and increasing acquisition speed.

[0019] 2) Enhance data integrity and accuracy: Automated operations reduce human error, such as incorrect file copying or data omissions, ensuring the completeness and accuracy of the acquired logs. Standardized naming and categorization facilitate quick location and retrieval of logs, increasing their usability and ensuring accurate and reliable analysis results.

[0020] 3) Enhanced transmission security: Employing SSL / TLS encryption technology prevents logs from being stolen or tampered with during transmission, protecting sensitive information. In complex network environments, this effectively avoids the risk of data leakage and ensures system security.

[0021] 4) Reduced Operations and Maintenance Costs: Reduced manual operations lower labor costs. Operations and maintenance personnel no longer need to spend significant time and effort on log acquisition and organization, allowing them to focus more on system optimization and troubleshooting. A unified management mechanism and standardized operating procedures reduce learning costs, enabling new operations and maintenance personnel to quickly get started and improving overall operational efficiency. Attached Figure Description

[0022] Figure 1 A flowchart illustrating the overall process of a multi-tool remote log aggregation and copying method based on disk adaptation; Figure 2 This is the control panel flowchart; Figure 3 This is a flowchart for the test side. Detailed Implementation

[0023] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] First, some terms and concepts in this invention are explained: 1) IP address range scanning: A network scanning technique that detects whether the hosts in a specified range are alive and what open ports are available by traversing a specified range of IP addresses, in order to discover devices in the network.

[0025] 2) Service discovery protocol: A protocol that allows devices to automatically discover each other and the services they provide on a network, which can help quickly locate hosts on the network that have specific tools installed.

[0026] 3) SSH (Secure Shell): A protocol used to provide secure remote login and other network services over insecure networks, ensuring data security through encrypted communication.

[0027] 4) SMB (Server Message Block): A network file sharing protocol that can be used to share resources such as files and printers between different operating systems, enabling command transmission and file interaction between the control end and the test end.

[0028] 5) FTP (File Transfer Protocol): A standard protocol for transferring files over a network, enabling the transfer of log files from the test end to the control end.

[0029] 6) SFTP (SSH File Transfer Protocol): A file transfer protocol built on top of the SSH protocol, providing encrypted file transfer functionality and ensuring secure log transmission.

[0030] 7) SSL / TLS (Secure Sockets Layer / Transport Layer Security): A protocol used to encrypt data in network communication, which can prevent data from being stolen or tampered with during transmission and ensure the security of log data transmission.

[0031] See Figures 1-3 The first aspect of this invention provides a method for remote log aggregation and copying using multiple tools based on disk adaptation, comprising the following steps: During the remote command sending and receiving phase, the control terminal automatically discovers test terminal hosts with tools installed on the local area network or within a specified range through network scanning; for different tools, the control terminal generates specific log copy commands according to the preset rules of their log storage location, and sends them to the corresponding test terminal hosts through network protocols; During the log copying and transmission phase, after receiving the log copy command, the test host parses the command content, confirms the path of the log file to be copied, and then selects the acquisition method to obtain the log based on the parsing result, executes the log copy operation, and copies the specified log file to a temporary storage location; then the test host encrypts the data in the temporarily stored log file and sends it back to the control end through the network transmission protocol; During the log processing and download phase, after receiving log files from various test hosts, the control terminal automatically categorizes and organizes them according to preset classification rules, and provides a log download interface for users.

[0032] In this embodiment, the control terminal is responsible for managing and coordinating the entire log acquisition process, including identifying the test host, sending log copy commands, and receiving and processing logs transmitted from the test host. The test host is deployed on various hosts with the tools installed, and is responsible for receiving commands sent by the control terminal, performing log copy operations, and transmitting the logs back to the control terminal.

[0033] The intelligent remote command sending and receiving mechanism allows the control terminal to identify test hosts: Through network scanning technology, the control terminal automatically discovers test hosts with relevant tools installed within the local area network or a specified range. It can quickly and accurately locate each test host using IP address range scanning, service discovery protocols, and other methods. It sends specific log copy commands: For different tools, the control terminal generates specific log copy commands based on preset rules regarding their log storage locations. These commands can accurately locate the log files of the tool in different disk locations. For example, for tool A with a known storage location pattern, the command can precisely specify a specific file path such as "C:\Program Files\ToolA\Logs\Current.log". The commands are then sent to the corresponding test hosts via network protocols (such as SSH, SMB, etc.).

[0034] Log copying and transmission on the test end involves receiving and executing commands: After receiving the log copy command from the control end, the test host parses the command content and confirms the path of the log file to be copied. Then, based on the parsing result, it retrieves the logs using various methods, including Windows / Linux special commands, specified path retrieval, specified tool interface retrieval, database query retrieval, message queue subscription retrieval, and automated script retrieval, and executes the log copy operation, copying the specified log file to a temporary storage location. Remote log transmission: The test host sends the temporarily stored log file back to the control host via network transmission protocols (such as FTP, SFTP, etc.). During transmission, data encryption technologies (such as SSL / TLS encryption) can be used to ensure the security of the log data and prevent data theft or tampering during transmission.

[0035] Control-end log processing and downloading, automated classification and organization: After receiving log files from various test terminals, the control-end host automatically classifies and organizes them according to preset classification rules. For example, classification can be based on tool type, date, log level, etc. Log files generated by the same tool at different times are organized into the same folder for easy user searching and management. Download interface provided: The control-end provides users with a convenient log download interface. Users can easily download the required log files to their local machine via a web interface, command-line tools, etc. During the download process, the system can provide progress display, resume interrupted downloads, and other functions to improve the user experience.

[0036] In some embodiments, the remote command sending and receiving phase further includes the following steps: Initialize control system resources and load configuration information; The control terminal initiates a TCP access probe on the test tool based on the IP address in the added host information. It sends a TCP connection request through a specific port to attempt to establish a connection with the test tool, thereby determining whether the tool is running. If the TCP access probe finds that the tool is not running, the control terminal sends a specific command to confirm its operating system type based on the response information returned by the test host. Then, it queries a specific location on the system based on the operating system type to determine if the tool is installed. If the tool is not detected to be installed, the control terminal sends an installation command based on the operating system type. If the tool is found to be installed but not running, the control terminal sends a startup command to start the tool according to the operating system type. The control terminal also records the identified IP address of the test host, the operating system type, and the tool's installation and startup status. After the tool starts, it determines the specific method of obtaining logs based on the characteristics of different tools and the preset log acquisition strategy, and generates a specific log copy command. Create a TCP socket using the network programming interface provided by the QT development framework, establish a TCP connection based on the IP address and specified port number of the test host, and after successful connection, serialize the log copy command and send it to the test host through the TCP socket.

[0037] In this embodiment, the process steps of the control terminal are as follows: 1) Control terminal startup: Initialize the control terminal system resources, load configuration information, and prepare for log acquisition and management.

[0038] 2) The control terminal performs a network scan to identify the test host: a) TCP access probe: Based on the IP address in the added host information, the control terminal initiates a TCP access probe against the test tool. By sending TCP connection requests to a specific port, it attempts to establish a connection with the test tool to determine if the tool is running. b) System judgment and tool detection: If the TCP access probe finds that the tool is not running, the control terminal sends a specific command and confirms its operating system type (Windows or Linux) based on the response information returned by the test terminal. After confirming the system type, it checks a specific location on the system (the registry or a specific installation directory for Windows systems, and the package management system installation records for Linux systems) to determine if the relevant tool is installed. c) Tool installation and startup: If the relevant tool is not detected to be installed, the control terminal sends an installation command according to the operating system type. For Windows systems, it calls a PowerShell script or a specific installer command to install; for Linux systems, it sends an installation command based on the package management tool used (such as apt-get, yum, etc.). If the tool is found to be installed but not running, the control terminal sends a startup command, calling the corresponding startup instruction to start the tool according to the operating system type. d) Information logging: Record detailed information of the identified test host, including IP address, operating system type, tool installation and startup status, etc., to provide a basis for sending log retrieval commands later.

[0039] 3) Determine the log acquisition method and generate instructions: Based on the characteristics of different tools and the preset log acquisition strategy, determine the specific method of log acquisition and generate corresponding detailed instructions. a) Windows system-specific command acquisition: If the tool is based on the Windows system and supports specific commands to acquire logs, such as acquiring system logs, generate the command "wevtutilqeSystem / rd:true / f:text>C:\Logs\SystemLog.txt" (assuming this command is used to acquire Windows system logs and save them to the specified path). b) Acquisition through a specified tool interface: If logs are acquired through a specified tool interface, such as a tool providing a Python API for acquiring logs, generate instructions containing information such as interface call parameters, such as "import my_tool_api; my_tool_api.get_logs (' / tmp / tool_logs.txt')" (assuming this code is used to acquire logs through the tool's Python API and save them to the specified file). c) Database Query Retrieval: For log retrieval via database query, determine the database type (e.g., MySQL, Oracle), host address, port, username, password, and other connection information, as well as the query statement, such as "SELECT * FROM logs_table WHERE log_type = 'error'" (assuming this statement is used to query error type logs from a table named logs_table). d) Other Retrieval Methods: Other retrieval methods follow similar steps to generate corresponding detailed instructions.

[0040] 4) The control end sends commands to the test end via network protocols: Regardless of whether the test end host's operating system is Linux or Windows, command transmission is based on the TCP protocol. Leveraging Qt's cross-platform capabilities, a TCP socket is created through Qt's network programming interface. A TCP connection is established based on the test end host's IP address and specified port number. After a successful connection, the command, including log retrieval methods and detailed instructions, is serialized and sent to the test end via the TCP socket.

[0041] In some embodiments, the log copying and transmission phase further includes the following steps: Start the network listening service on the test host and configure the client parameters; The network communication module based on the QT development framework is used to receive and save log copy commands; The log copy command is read, and the acquisition method and related parameter information are extracted according to the preset command format parsing rules. The log is then retrieved based on the acquisition method. The system retrieves and formats the current time, and determines the host type by reading system configuration files or environment variables. It then names the log files according to naming rules and selects a compression format to compress the log files, resulting in a compressed log archive. Based on the control terminal configuration, select the network transmission protocol, configure the protocol parameters, encrypt the log compressed package, and then transmit it back to the control terminal.

[0042] In this embodiment, the process steps of the test end are as follows: 1) The test end starts and listens: The test end starts a network listening service, configures client parameters based on the TCP protocol and QT's cross-platform features, and waits for the control end to send commands. This step initializes the basic environment for communication between the test end and the control end, ensuring that it can receive instructions from the control end.

[0043] 2) Receive and save commands: Use a network communication module based on QT to receive commands sent by the control terminal, including log retrieval methods and detailed instructions, and save them to a specified location. This ensures accurate reception and storage of commands, preparing for subsequent parsing and execution.

[0044] 3) Parse commands to determine the operation: Read the contents of the received command file, and extract the log acquisition method and related parameter information according to the preset command format parsing rules. Determine the log acquisition method, such as whether it is obtained through special Windows / Linux commands, specified paths, specified tool interfaces, database queries, message queue subscriptions, or automated script acquisition.

[0045] 4) Obtain logs according to the selected method.

[0046] 5) Log Naming and Compression: a) Naming Information Acquisition: Obtain the current time and format it as YYYYMMDDHHMMSS. Simultaneously, determine the host type by reading system configuration files or environment variables. b) Log Naming: Name the log file according to the naming rules, in a format similar to "20240101120000_dev_host_log.txt". c) Compression Operation: Determine the compression format. If zip format is selected, use a compression library (such as the zipfile library) to compress the log file into a compressed archive named "20240101120000_dev_host_log.zip"; if tar.gz format is selected, use the tarfile library for compression.

[0047] 6) Transmitting the compressed log package: Based on the control terminal configuration, select a suitable network transmission protocol (such as FTP or SFTP), and configure the relevant parameters of the transmission protocol, such as the server address, port number, username, and password for the FTP protocol. Use the corresponding protocol library (such as ftplib for the FTP protocol, and paramiko's SFTP client for the SFTP protocol) to transmit the compressed log package back to the control terminal host. During the transmission process, data encryption technology (such as SSL / TLS encryption) is used to ensure the security of the log data.

[0048] In some embodiments, the log processing and downloading phase further includes the following steps: The control unit starts network monitoring, configures server-side parameters according to the selected network transmission protocol, and uses the protocol library to receive and save log compressed packages. Determine the compression format of the log archive and use the corresponding decompression library to perform the decompression operation, extracting the log files in the log archive to the specified directory. Read the preset classification rules, determine the classification dimensions, traverse the decompressed log files, and move the log files to the corresponding folders according to the classification rules based on the log information in the log files. A download button is provided for each log file. When the user clicks the download button, the program calls the underlying download function module, obtains the storage path of the selected log file, starts the download process, and provides a resume function. When the download is interrupted, the downloaded part is recorded, and the download will start from the breakpoint when it resumes.

[0049] In some embodiments, the SSL / TLS encryption protocol is used for data encryption.

[0050] In some embodiments, the acquisition methods include Windows / Linux special command acquisition, acquisition via specified path, acquisition via specified tool interface, database query acquisition, message queue subscription acquisition, and script automation acquisition.

[0051] In this embodiment, the specific implementation steps for each acquisition method are as follows: a) Windows / Linux special command acquisition: If it is determined that the log is to be acquired through Windows or Linux special commands, the operating system type is first determined. If it is a Windows system, the Windows system command-line tool is called (such as calling the command through Python's subprocess module), and the corresponding command is executed, such as "wevtutil qeSystem / rd:true / f:text>C:\Logs\SystemLog.txt", and the command execution result is saved as a log file. If it is a Linux system, a similar method is used to call Linux commands, such as "journalctl -u my_service> / tmp / my_service_log.txt", to obtain the corresponding log information and save it as a log file.

[0052] b) Obtaining from a specified path: Extract the specified path information from the command. Determine the operating system type. For Windows systems, use file operation functions (such as functions related to Python's `os` module), such as "xcopy C:\Program Files\ToolX\Logs*.* D:\Temp\Logs\ / E / Y" to copy the log files from the specified path to a temporary storage location. For Linux systems, use the command "cp -r / path / to / source_logs / tmp / temp_logs" to complete the log file copy.

[0053] c) Obtaining the specified tool interface: Extract the interface information and parameters from the command. Based on the interface type provided by the tool, load the corresponding interface library (if the tool provides a Python API, import the relevant module), call the interface function or method, such as "import my_tool_api; my_tool_api.get_logs (' / tmp / tool_logs.txt')", obtain the logs and save them to the specified file.

[0054] d) Database Query Retrieval: Extract database connection information (such as database type, host address, port, username, password, etc.) and query statements from the command. Based on the database type, import the corresponding database connection library (e.g., use the mysql.connector library for MySQL), establish a connection with the database using the connection information, execute the query statements, and retrieve relevant log data. Organize the query results and save them to a log file, such as writing the query results to the file " / tmp / database_log.txt".

[0055] e) Message Queue Subscription Retrieval: Extract message queue-related information from the command (such as message queue server address, port, queue name, authentication information, etc.). Based on the message queue type, import the corresponding client library (e.g., pika library for RabbitMQ message queues), connect to the message queue server using the relevant information, declare the queue, and set a callback function to handle received messages. In the callback function, organize the received log messages and save them to a log file, such as " / tmp / queue_log.txt".

[0056] f) Automated script retrieval: Extract the script path and execution parameters from the command. Determine the script language type; if it's a Python script, execute the script using a script interpreter (such as the `python` command), e.g., "python / path / to / your_script.py arg1 arg2". Internally, the script follows pre-defined logic, potentially involving system command calls, file operations, data processing, etc., retrieving and organizing log information, and finally generating a log file.

[0057] A second aspect of the present invention provides: a multi-tool remote log aggregation and copying system based on disk adaptation, used to implement any of the above-mentioned multi-tool remote log aggregation and copying methods based on disk adaptation, comprising: The remote command sending and receiving module is used by the control terminal to automatically discover test terminal hosts with tools installed on the local area network or within a specified range through network scanning; the control terminal generates specific log copy commands for different tools according to preset rules for their log storage locations, and sends them to the corresponding test terminal hosts through network protocols; The log copy and transmission module is used to parse the log copy command after the test host receives it, confirm the path of the log file to be copied, select the acquisition method to obtain the log based on the parsing result, and perform the log copy operation to copy the specified log file to a temporary storage location. Then, the test host encrypts the data in the temporarily stored log file and sends it back to the control end through the network transmission protocol. The log processing and download module is used to automatically classify and organize log files received from various test host terminals on the control terminal according to preset classification rules, and provide users with a log download interface.

[0058] A third aspect of the present invention provides: a computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, they implement any of the above-described methods for multi-tool remote log aggregation and copying based on disk adaptation.

[0059] A fourth aspect of the present invention provides: a computer program product containing instructions, which, when run on a terminal, causes the terminal to execute any of the above-described methods for multi-tool remote log aggregation and copying based on disk adaptation.

[0060] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A multi-tool remote log aggregation and copying method based on disk adaptation, characterized in that: Includes the following steps: During the remote command sending and receiving phase, the control terminal automatically discovers test terminal hosts with tools installed on the local area network or within a specified range through network scanning; for different tools, the control terminal generates specific log copy commands according to the preset rules of their log storage location, and sends them to the corresponding test terminal hosts through network protocols; During the log copying and transmission phase, after receiving the log copy command, the test host parses the command content, confirms the path of the log file to be copied, and then selects the acquisition method to obtain the log based on the parsing result, executes the log copy operation, and copies the specified log file to a temporary storage location; then the test host encrypts the data in the temporarily stored log file and sends it back to the control end through the network transmission protocol; During the log processing and download phase, after receiving log files from various test hosts, the control terminal automatically classifies and organizes them according to preset classification rules and provides a log download interface for users. The remote command sending and receiving phase also includes the following steps: Initialize control system resources and load configuration information; The control terminal initiates a TCP access probe on the test tool based on the IP address in the added host information. It sends a TCP connection request through a specific port to attempt to establish a connection with the test tool, thereby determining whether the tool is running. If the TCP access probe finds that the tool is not running, the control terminal sends a specific command to confirm its operating system type based on the response information returned by the test host. Then, it queries a specific location on the system based on the operating system type to determine if the tool is installed. If the tool is not detected to be installed, the control terminal sends an installation command based on the operating system type. If the tool is found to be installed but not running, the control terminal sends a startup command to start the tool according to the operating system type. The control terminal also records the identified IP address of the test host, the operating system type, and the tool's installation and startup status. After the tool starts, it determines the specific method of obtaining logs based on the characteristics of different tools and the preset log acquisition strategy, and generates a specific log copy command. Create a TCP socket using the network programming interface provided by the QT development framework, establish a TCP connection based on the IP address and specified port number of the test host, and after successful connection, serialize the log copy command and send it to the test host through the TCP socket.

2. The multi-tool remote log aggregation and copying method based on disk adaptation according to claim 1, characterized in that: The log copying and transmission phase also includes the following steps: Start the network listening service on the test host and configure the client parameters; Use the network communication module based on the QT development framework to receive and save log copy commands; The log copy command is read, and the acquisition method and related parameter information are extracted according to the preset command format parsing rules. The log is then retrieved based on the acquisition method. The system retrieves and formats the current time, and determines the host type by reading system configuration files or environment variables. It then names the log files according to naming rules and selects a compression format to compress the log files, resulting in a compressed log archive. Based on the control terminal configuration, select the network transmission protocol, configure the protocol parameters, encrypt the log compressed package, and then transmit it back to the control terminal.

3. The multi-tool remote log aggregation and copying method based on disk adaptation according to claim 1, characterized in that: The log processing and downloading stage also includes the following steps: The control unit starts network monitoring, configures server-side parameters according to the selected network transmission protocol, and uses the protocol library to receive and save log compressed packages. Determine the compression format of the log archive and use the corresponding decompression library to perform the decompression operation, extracting the log files in the log archive to the specified directory. Read the preset classification rules, determine the classification dimensions, traverse the decompressed log files, and move the log files to the corresponding folders according to the classification rules based on the log information in the log files. A download button is provided for each log file. When the user clicks the download button, the program calls the underlying download function module, obtains the storage path of the selected log file, starts the download process, and provides a resume function. When the download is interrupted, the downloaded part is recorded, and the download will start from the breakpoint when it resumes.

4. The multi-tool remote log aggregation and copying method based on disk adaptation according to claim 1, characterized in that: Data is encrypted using the SSL / TLS encryption protocol.

5. The multi-tool remote log aggregation and copying method based on disk adaptation according to any one of claims 1-4, characterized in that: The acquisition methods include Windows / Linux special commands, specified paths, specified tool interfaces, database queries, message queue subscriptions, and automated script acquisition.

6. A multi-tool remote log aggregation and copying system based on disk adaptation, characterized in that: The method for implementing the multi-tool remote log aggregation and copying method based on disk adaptation as described in any one of claims 1-5 includes: The remote command sending and receiving module is used by the control terminal to automatically discover test terminal hosts with tools installed on the local area network or within a specified range through network scanning; the control terminal generates specific log copy commands for different tools according to preset rules for their log storage locations, and sends them to the corresponding test terminal hosts through network protocols; The log copy and transmission module is used to parse the log copy command after the test host receives it, confirm the path of the log file to be copied, select the acquisition method to obtain the log based on the parsing result, and perform the log copy operation to copy the specified log file to a temporary storage location. Then, the test host encrypts the data in the temporarily stored log file and sends it back to the control end through the network transmission protocol. The log processing and download module is used to automatically classify and organize log files received from various test hosts on the control end according to preset classification rules, and provide users with a log download interface.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the multi-tool remote log aggregation and copying method based on disk adaptation as described in any one of claims 1-5.

8. A computer program product containing instructions, characterized in that: When the computer program product is run on a terminal, the terminal executes the multi-tool remote log aggregation and copying method based on disk adaptation as described in any one of claims 1-5.

Images