Server inspection system based on SSH protocol

Abstract

The invention discloses a server inspection system based on an SSH protocol, and the system is characterized in that an acquisition task scheduling module is used for obtaining task execution configuration information, and determining an acquisition target; the server information acquisition module or the database information acquisition module is used for acquiring a data acquisition script according to an operating system type or a database type of an acquisition target, and establishing remote connection with a target remote server or database through an SSH protocol; the target remote server or the database executes the data acquisition script, obtains text data and returns the text data; the data cleaning module is used for performing data cleaning on the collected text data to obtain structured data and storing the structured data in a database; the report output module is used for generating an inspection report according to the structured data; and the data display module is used for visually displaying the inspection report through a Web page. According to the server inspection system provided by the invention, the server can be scheduled and monitored in a unified manner, and the server inspection system has relatively high flexibility.

Description

Technical Field

[0001] This application relates to the field of server inspection technology, specifically to a server inspection system based on the SSH protocol. Background Technology

[0002] With the continuous development of information technology, enterprise IT infrastructure is becoming increasingly complex. Especially with the widespread application of technologies such as cloud computing, big data, and artificial intelligence, the management and monitoring of server resources have become particularly important. Key indicators such as server CPU, memory, and hard disk space are core elements for ensuring the normal operation of the system. However, traditional server management often cannot obtain server health status in real time and efficiently, and it is difficult to provide early warnings of potential failures.

[0003] Currently, most monitoring systems used by enterprises often have the following problems:

[0004] (1) Insufficient real-time performance: Traditional monitoring systems have high latency and cannot reflect the usage of server resources in a timely manner, making it difficult to achieve rapid response.

[0005] (2) Incomplete data: Existing monitoring tools may only focus on certain basic indicators, such as CPU utilization and memory usage, while ignoring the monitoring of other important resources such as hard disk space and network bandwidth.

[0006] (3) Difficulty in making accurate predictions and fault diagnosis: Although most monitoring platforms can collect data in real time, they lack intelligent data analysis, cannot provide early warnings of server faults, and cannot predict potential problems through historical data analysis.

[0007] (4) High reliance on manual management: Many small and medium-sized enterprises still rely on manual inspection and maintenance of servers due to budget constraints. They lack automated operation and maintenance management tools, resulting in low work efficiency and easy errors.

[0008] To address the aforementioned technical problems, patent document CN118760566A discloses a server inspection method, apparatus, device, and storage medium. This method uses a detection script pre-stored in the server to be inspected to inspect all connected servers. However, this technical solution may have the following drawbacks:

[0009] (1) Reliance on pre-stored scripts on servers: This technical solution requires that the detection scripts be pre-stored on each server to be inspected. In this way, if the server has not deployed the script in advance or the script has problems (e.g., the script is corrupted or not updated), the inspection task cannot be executed, resulting in inspection failure or incompleteness;

[0010] (2) Difficulty in updating and maintaining: If it is necessary to modify or update the inspection script (e.g., add new detection items or fix vulnerabilities), the update must be performed individually on each server. This method is particularly troublesome in large-scale data centers or cloud environments, making centralized management impossible and increasing the complexity and cost of operation and maintenance;

[0011] (3) Security issues: Storing inspection scripts on the server may pose certain security risks. If attackers can obtain the scripts, they may be able to bypass the inspection by modifying or replacing them, causing the system to fail to detect problems or security vulnerabilities in a timely manner;

[0012] (4) Server resource consumption: Storing and executing the detection script may consume certain server resources. Especially when server resources are scarce, the execution of the inspection script may affect the normal operation of the server and even lead to performance degradation.

[0013] (5) Unified scheduling and monitoring: If the detection scripts are stored on each server and need to be executed independently by each server, the scheduling and execution process of the inspection tasks may be scattered and difficult to monitor, making it impossible to achieve unified global scheduling and efficient management.

[0014] (6) Poor scalability: As the number of servers to be inspected increases, this solution may encounter bottlenecks when scaling up. Each new server requires the deployment of the detection script in advance, which is not flexible and efficient enough for large-scale deployment.

[0015] In summary, this approach has certain limitations in terms of flexibility, efficiency, security, and maintainability, and its performance is particularly poor in large-scale, highly dynamic environments. Summary of the Invention

[0016] To address this issue, this application provides a server inspection system based on the SSH protocol, which solves the problems of existing server inspection methods that cannot be uniformly scheduled and monitored, and have poor flexibility.

[0017] To achieve the above objectives, this application provides the following technical solution:

[0018] A server inspection system based on the SSH protocol includes:

[0019] The task scheduling module is used to acquire task execution configuration information, generate acquisition instructions periodically or triggered according to the task execution configuration information, and determine the acquisition target; the acquisition target includes remote servers and database servers;

[0020] The server information acquisition module is used to obtain a data acquisition script based on the operating system type of the target, and establish a remote connection with the target remote server via the SSH protocol; the target remote server executes the data acquisition script, obtains text data, and returns the text data to the server information acquisition module;

[0021] The database information acquisition module is used to obtain a data acquisition script based on the database type of the target database and establish a remote connection with the target database via the SSH protocol; the target database executes the data acquisition script, acquires text data, and returns the text data to the database information acquisition module.

[0022] The data cleaning module is used to clean the text data collected by the server information collection module or the database information collection module according to a pre-set data processing script, obtain structured data, and save it to the database.

[0023] The report output module is used to generate inspection reports based on structured data;

[0024] The data display module is used to visualize the inspection report through a web page.

[0025] Optionally, when the server information acquisition module or the database information acquisition module establishes a remote connection with the target remote server or the target database via the SSH protocol, it is implemented using the paramiko library in Python.

[0026] Optionally, the text data collected by the server information collection module or the database information collection module includes CPU utilization, memory usage, and hard disk usage.

[0027] Optionally, the data cleaning module uses Python's pandas library for data cleaning.

[0028] Optionally, the data cleaning module includes processing missing data, outlier detection, data formatting, data standardization, and data type conversion during data cleaning.

[0029] Optionally, the database is an SQL database or a NoSQL database.

[0030] Optionally, the inspection report generated by the report output module is in PDF, Excel, or HTML format.

[0031] Optionally, the report output module uses the ReportLab library or the WeasyPrint library when generating inspection reports based on structured data.

[0032] Optionally, the web page uses the Flask framework.

[0033] Optionally, the data display module uses Plotly, D3.js, or Matplotlib libraries to generate charts when performing visualization.

[0034] Compared with the prior art, this application has at least the following beneficial effects:

[0035] This application provides a server inspection system based on the SSH protocol. A task scheduling module acquires task execution configuration information, generates collection instructions periodically or triggered based on the configuration information, and determines the collection target. A server information collection module or database information collection module acquires a data collection script based on the operating system type or database type of the collection target and establishes a remote connection with the target remote server or database via the SSH protocol. The target remote server or database executes the data collection script, acquires text data, and returns the text data. A data cleaning module cleans the text data acquired by the server information collection module or database information collection module according to a pre-set data processing script, obtaining structured data and saving it to the database. A report output module generates an inspection report based on the structured data. A data display module visualizes the inspection report through a web page. The server inspection system provided in this application can uniformly schedule and monitor servers, has high flexibility, and is suitable for large-scale, highly dynamic environments. Attached Figure Description

[0036] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0037] Figure 1 A schematic diagram of the structure of a server inspection system based on the SSH protocol provided in this application;

[0038] Figure 2 The data processing flowchart of the data acquisition task scheduling module provided in this application;

[0039] Figure 3 The data processing flowchart for the server information acquisition module provided in this application;

[0040] Figure 4 The data processing flowchart for the database information acquisition module provided in this application;

[0041] Figure 5 The data processing flowchart for the data cleaning module provided in this application. Detailed Implementation

[0042] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "comprising," "including," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0044] The terms used in this application, such as "upper," "lower," "left," "right," and "middle," are generally used to indicate the general relative positional relationship for the purpose of intuitive understanding by referring to the accompanying drawings, and are not absolute limitations on the positional relationship in the actual product.

[0045] This application aims to provide a server inspection system based on the SSH protocol. It queries the server status by remotely executing system commands, automatically parses the query results and stores them in the database, and combines automatically generated inspection reports with visualization functions implemented by Flask to help users efficiently, flexibly and in real time monitor and manage servers, promptly discover and resolve potential problems, and improve operation and maintenance efficiency and system stability.

[0046] Please see Figure 1 This application provides a server inspection system based on the SSH protocol, comprising:

[0047] The task scheduling module is used to obtain task execution configuration information, generate collection instructions periodically or triggered according to the task execution configuration information, and determine the collection targets; the collection targets include remote servers and database servers.

[0048] Specifically, the data processing procedure of the task scheduling module is as follows (e.g.) Figure 2 As shown):

[0049] Step 1: After the task scheduling module starts, it reads the task execution configuration information, which includes: task execution time and data collection target;

[0050] Step 2: After reading the task execution configuration information, generate collection commands (e.g., top, df, free, etc.) and determine the collection target. The task scheduling engine starts; after the task scheduling engine starts, it collects data from the target server or database at an appropriate time.

[0051] Step 3: After data collection is completed, the data cleaning module is entered. The module judges the collection results and returns the judgment results to the task scheduling engine, which decides whether the current collection is to end.

[0052] The data collection task scheduling module provided in this application performs server inspections by setting scheduled tasks and automated scripts. This is achieved through Python's schedule library, APScheduler, or cron jobs under Linux.

[0053] In summary, the data acquisition task scheduling module provided in this application has the following functions:

[0054] (1) Responsible for the scheduling and management of data collection tasks, ensuring that timed or triggered tasks can be executed according to predetermined rules; specifically, task scheduling tools such as Python's schedule library, APScheduler, or Linux's cron task can be used to implement timed task scheduling and parallel execution.

[0055] (2) Supports scheduled execution of tasks (e.g., scheduled execution of data collection tasks every day);

[0056] (3) It can perform tasks in parallel or distributed manner, thereby improving efficiency;

[0057] (4) Supports functions such as task failure retry and logging to ensure high task reliability.

[0058] The server information acquisition module is used to obtain the data acquisition script based on the operating system type of the target and establish a remote connection with the target remote server via the SSH protocol; the target remote server executes the data acquisition script, obtains text data, and returns the text data to the server information acquisition module;

[0059] Specifically, the data processing procedure of the server information acquisition module is as follows (e.g.) Figure 3 As shown):

[0060] Step 1: After receiving the collection instruction from the collection task scheduling module, the server information collection module begins to collect information from the target server;

[0061] Step 2: After the data collection task starts, the server information collection module will read the target remote server information and call the data collection script pre-installed in the system based on the target remote server information to achieve compatibility with different operating systems;

[0062] Step 3: After assembling the data acquisition script, establish a remote connection with the target remote server and execute the data acquisition script directly on the target remote server via the SSH protocol;

[0063] Step 4: The server information collection module will receive the script execution results and transmit the collection results to the data cleaning module, thus ending this collection.

[0064] The server information acquisition module provided in this application interacts with a remote server via Python scripts to periodically obtain the server's operating status. This technology utilizes various Python libraries (such as paramiko for SSH communication and psutil for system monitoring) to achieve remote management and status monitoring of the remote server.

[0065] In summary, the server information collection module provided in this application has the following functions:

[0066] (1) Responsible for connecting to remote servers via the SSH protocol, executing preset commands, and obtaining server status information, such as CPU usage, memory usage, disk space, etc. Specifically, the paramiko library of Python or other SSH tools can be used for remote connection and command execution.

[0067] (2) It can perform regular or real-time monitoring and information collection on different remote servers; during collection, it achieves compatibility with different query systems through pre-made programs;

[0068] (3) Multiple servers need to be managed to ensure the reliability and real-time performance of the data collection;

[0069] (4) Data collection can be performed through scheduled tasks or trigger conditions.

[0070] The database information acquisition module is used to obtain the data acquisition script according to the database type of the target database and establish a remote connection with the target database via the SSH protocol; the target database executes the data acquisition script, obtains text data, and returns the text data to the database information acquisition module;

[0071] Specifically, the data processing procedure of the database information acquisition module is as follows (e.g.) Figure 4 As shown):

[0072] Step 1: After receiving the collection instruction from the collection task scheduling module, the database information collection module begins to collect information from the database server;

[0073] Step 2: After the data collection task starts, the database information collection module will read the target database information and call the data collection script pre-installed in the system according to the target database information to achieve compatibility with different operating systems;

[0074] Step 3: After assembling the data acquisition script, establish a remote connection with the target database server and execute commands directly on the target database server via the SSH protocol; the database server will automatically read all table information of the specified database, record the data volume, and return the running results;

[0075] Step 4: The database information acquisition module receives the command execution results and transmits the acquisition results to the data cleaning module, thus ending this acquisition.

[0076] In summary, the database information acquisition module provided in this application has the following functions:

[0077] (1) Complete the database application inspection function;

[0078] (2) Collect the database cluster's operating status;

[0079] (3) Collect information such as server running status and tablespace size;

[0080] (4) Collect data on changes in the amount of data in the data table;

[0081] (5) Collect database backup information.

[0082] The database information acquisition module provided in this application can use Python and database interface libraries (such as SQLAlchemy or pymysql) to perform data writing operations; it may also involve database performance optimization, especially when processing large amounts of data.

[0083] The data cleaning module is used to clean the text data collected by the server information collection module or the database information collection module according to the pre-set data processing script, so as to obtain structured data (e.g., JSON, dictionary or list format) and save it to the database.

[0084] Specifically, the data cleaning module includes handling missing data, outlier detection, data formatting, data standardization, and data type conversion during data cleaning. The process is as follows: Figure 5 As shown:

[0085] Handling Missing Data: Check the dataset for null or missing values. Missing Value Imputation: Choose an appropriate imputation method based on the data type and requirements, such as mean imputation, median imputation, most frequent value imputation, or use advanced methods such as interpolation or regression. Removing Missing Values: If the proportion of missing data is large and cannot be reasonably imputed, consider removing the relevant rows or columns.

[0086] Outlier detection: Statistical methods (e.g., box plots, Z-scores) or machine learning methods are used to detect outlier data points. Outlier correction or removal: Depending on the specific circumstances, a decision is made on whether to correct or remove these outliers to avoid impacting subsequent analysis.

[0087] Data formatting: Ensure that the data conforms to the predetermined format requirements, such as consistent date format and consistent text case.

[0088] Data standardization: For numerical data, standardization or normalization is performed to ensure the data is within a certain range and to avoid scale differences affecting subsequent analysis. Unit consistency: Ensure that the units of all indicators in the data are consistent; for example, convert all monetary units to a unified unit.

[0089] Data type conversion: Converting data types according to data requirements. For example, converting string-type dates to date types, or converting numeric data to categorical variables.

[0090] The data cleaning process will be reported back to the task scheduling engine. If there are missing or excessive errors, the data collection program will be re-initiated. If the data is complete, the cleaned data will be output, and the cleaned structured data will be saved to the database.

[0091] The data cleaning module provided in this application stores the acquired status data in a database for subsequent querying, analysis, and report generation. This typically uses SQL databases (such as MySQL or PostgreSQL) or NoSQL databases (such as MongoDB) to store and manage large amounts of monitoring data. Data parsing techniques are used to process the raw status data and perform storage, querying, and analysis as needed.

[0092] In summary, the data cleaning module provided in this application has the following functions:

[0093] (1) Responsible for cleaning and preprocessing the collected raw data; specifically, the pandas library of Python can be used for data processing, cleaning and transformation.

[0094] (2) It may filter and correct abnormal or invalid data;

[0095] (3) Format and standardize the collected data to ensure that the data stored in the database conforms to the expected format;

[0096] (4) Fill in or remove missing data to maintain data quality.

[0097] The report output module is used to generate inspection reports based on structured data.

[0098] Specifically, the report output module is responsible for generating inspection reports based on the collected and cleaned data. It supports generating reports on demand, which can be based on statistical data, server status, or detailed information on fault alarms for a certain period of time.

[0099] The inspection reports generated by the report output module can be in PDF, Excel, or HTML format for easy viewing and archiving by users. For PDF reports, libraries such as ReportLab or WeasyPrint can be used; for Excel reports, the pandas to_excel() method can be used; and for HTML reports, the Jinja2 template engine can be used, combined with Flask for dynamic generation.

[0100] The data display module is used to visualize the inspection report through a web page.

[0101] Specifically, the data visualization module is responsible for visualizing the collected data through a web page, helping users intuitively understand the server's status. It displays server health and performance data (such as CPU, memory, and network bandwidth) through charts (JavaScript charting libraries, such as Chart.js and D3.js), dashboards, and other formats. It supports real-time data display and can even set thresholds for key indicators; when an indicator exceeds a warning value, the system can provide an alert.

[0102] Specifically, the data visualization module uses the Flask framework to develop a web application. Flask is a lightweight web framework used to build the platform's visualization interface. Through the web application functions provided by Flask, users can view the server status in real time and interact with it through a graphical interface; visualizations can use libraries such as Plotly, D3.js, or Matplotlib to generate charts; and JavaScript and front-end technologies (such as React and Vue.js) are used to enhance the user experience.

[0103] In addition, to display server status and monitoring data, front-end technologies (such as HTML, CSS, JavaScript, and visualization libraries D3.js or Chart.js) are used to display real-time and historical data charts, helping operations and maintenance personnel to intuitively understand the health status of the servers.

[0104] In summary, the server inspection system based on the SSH protocol provided in this application has the following functions:

[0105] (1) Remote server monitoring and command execution: Communicate with remote servers via SSH protocol, execute specified system commands periodically or as needed, and obtain server resource usage and health status (e.g., CPU load, memory usage, hard disk space, etc.);

[0106] (2) Data parsing and storage: After executing the command, the system parses the query results returned by the server. After extracting and formatting the data using a Python script, the parsed results are stored in a database (such as MySQL, PostgreSQL, etc.) to ensure long-term data preservation and subsequent analysis;

[0107] (3) Inspection Report Generation: Another Python script generates an inspection report for the server based on stored historical data. This report includes the server's health status, performance indicators, potential problems, and optimization suggestions, making it easier for maintenance personnel to understand the system status and identify and resolve potential problems in advance;

[0108] (4) Visualization: Using the Flask framework, a web application is provided to present real-time data and historical reports of the server status. Flask intuitively displays server performance metrics through dynamic charts, data tables, and other formats, making it easy for users to monitor the server in real time.

[0109] The server inspection system based on the SSH protocol provided in this application has the following advantages:

[0110] (1) Automated monitoring and query: The system establishes a connection with the remote server via the SSH protocol and automatically queries the server's status information, reducing manual intervention and improving the efficiency and accuracy of monitoring.

[0111] (2) Data parsing and storage: The server status results obtained by querying are automatically stored in the database after being parsed by Python scripts, ensuring long-term storage and management of data, which facilitates data analysis and troubleshooting in the future.

[0112] (3) Inspection report generation: Another script generates server inspection reports, which can automatically generate detailed reports, helping maintenance personnel to quickly understand the server's operating status and reducing the time and manpower costs of report generation.

[0113] (4) Visualization: The Flask framework is used to visualize the server status, transforming the data into an intuitive graphical interface to help users keep track of the server’s health status in real time, thereby improving user experience and decision-making efficiency.

[0114] (5) Improve operation and maintenance efficiency: Through automation, data storage and visualization technologies, the system significantly improves the efficiency and response speed of server operation and maintenance, avoids errors in manual operation, and reduces the risk of system failure.

[0115] (6) Enhance decision support: Provides real-time server status display and historical data reports, which makes it easier for administrators to make more scientific operation and maintenance decisions and improves the stability and security of the system.

[0116] The server inspection system provided in this application integrates functions such as automatic query, data parsing, report generation, and visualization. It can uniformly schedule and monitor servers, realize comprehensive monitoring and analysis of server status, has high flexibility, is suitable for large-scale and highly dynamic environments, and can provide strong technical support for operation and maintenance management.

[0117] The technical features of the above embodiments can be combined in any way (as long as there is no contradiction in the combination of these technical features). For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A server inspection system based on the SSH protocol, characterized in that, include: The task scheduling module is used to acquire task execution configuration information, generate acquisition instructions periodically or triggered according to the task execution configuration information, and determine the acquisition target. The data collection targets include remote servers and database servers; The server information acquisition module is used to obtain a data acquisition script based on the operating system type of the target, and establish a remote connection with the target remote server via the SSH protocol; the target remote server executes the data acquisition script, obtains text data, and returns the text data to the server information acquisition module; The database information acquisition module is used to obtain a data acquisition script based on the database type of the target database and establish a remote connection with the target database via the SSH protocol; the target database executes the data acquisition script, acquires text data, and returns the text data to the database information acquisition module. The data cleaning module is used to clean the text data collected by the server information collection module or the database information collection module according to a pre-set data processing script, obtain structured data, and save it to the database. The report output module is used to generate inspection reports based on structured data; The data display module is used to visualize the inspection report through a web page.

2. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The server information acquisition module or the database information acquisition module uses the paramiko library in Python to establish a remote connection with the target remote server or target database via the SSH protocol.

3. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The text data collected by the server information collection module or the database information collection module includes CPU usage, memory usage, and hard disk usage.

4. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The data cleaning module uses the pandas library in Python for data cleaning.

5. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The data cleaning module includes handling missing data, outlier detection, data formatting, data standardization, and data type conversion during data cleaning.

6. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The database can be an SQL database or a NoSQL database.

7. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The inspection report generated by the report output module is in PDF, Excel, or HTML format.

8. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The report output module uses the ReportLab library or WeasyPrint library to generate inspection reports based on structured data.

9. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The web page uses the Flask framework.

10. The server inspection system based on the SSH protocol according to claim 1, characterized in that, The data display module uses Plotly, D3.js, or Matplotlib libraries to generate charts when performing visualization.

Images