Network Range Log File Collection Agent Gateway, Collection System and Method

Abstract

The present invention discloses a network range log file collection proxy gateway, a collection system and a method. The present invention adds a collection proxy gateway to the network range scenario, and the proxy gateway communicates with the collection servers and file servers inside and outside the range scenario respectively; the proxy gateway is provided with an encryption module, an instruction forwarding module, a policy configuration module and a file transfer module; the instruction forwarding module receives and parses the collection instructions sent by the collection server, and according to the metadata information in the collection instructions, distributes the collection instructions to the probes on the specified virtual machines or physical machines; at the same time, collects the results of the probes executing the instructions and feeds them back to the collection server; the policy configuration module queries the policy configuration information from the collection server according to the policy configuration ID and stores it locally in the proxy gateway for the probes to query; the file transfer module caches the log files collected by the probes. The present invention can solve the problem of network isolation, enhance the probe ability and reduce the pressure on the collection server.

Description

Technical Field

[0001] The invention relates to a network range log file collection agent gateway, a collection system and a method, and belongs to the field of network technology. Background Art

[0002] Cyber ​​Range is a technology or product based on virtualization technology that simulates and reproduces the operating status and operating environment of network architecture, system equipment, and business processes in real cyberspace, so as to more effectively realize learning, research, testing, competition, exercises and other behaviors related to network security, thereby improving the network security confrontation level of personnel and institutions.

[0003] In a real network range environment, there will be many machines running at the same time, including virtual machines, physical machines, etc., which will generate a large number of log files in real time. These log files record the operating status information of the system and applications, which is of great value for analyzing or reviewing security incidents, system stability, system risks, vulnerabilities, etc. in the network range.

[0004] Since most of the devices in the network range are in a closed network environment and are not interoperable with the outside network, it is very inconvenient to obtain log files on some of the machines. At present, the main way to obtain log files on a machine is as follows: Figure 1 The solution shown. Deploy a scenario management system to manage the network range scenario and the settings of the machines in the scenario, as well as initialization, networking, and destruction operations. When the scenario management system creates a network range scenario, it will install probes on each machine in the scenario to collect machine information and report it to the collection server. When the scenario management system starts the scenario, it will configure the log files or directory paths that the probes need to monitor. In this way, after the probes are started, the corresponding file monitoring information will be collected and uploaded to the collection server. Through the scenario management system, users can query the file information under a certain machine in a certain scenario from the collection server, including file name, path, size, status, permissions, etc. In addition to the network range scenario, a remote desktop gateway system needs to be deployed separately, which will manage the remote connections of all machines in all network scenarios. After the user creates and initializes the scenario through the scenario management system, the remote connection protocol, account password or secret key of all machines in the scenario, and the scenario, networking topology, and other information need to be pushed to the remote desktop gateway system for storage. By logging in to the remote desktop gateway, users can select and log in to a machine in a certain scenario. Users can use the download tool provided by the remote desktop gateway to download the specified log file in the machine to the user's local machine.

[0005] The existing solution has the following disadvantages: 1. A desktop gateway system needs to be deployed separately, which increases the server resource overhead. 2. After the network range scene is initialized, the scene management system needs to push the access protocol, account password, secret key and other information of all machines in the scene to the desktop gateway. If the scene information or the access information of the machine in it changes in the middle, such as the login account or password of a machine in the scene, it must also be pushed to the remote desktop gateway system in real time, which increases the complexity of the system and increases the difficulty of implementation and maintenance. 3. The probes installed inside the virtual machine or physical machine in the network range scene communicate directly with the collection server. If the network scene as a whole is in a closed network, this communication will be blocked, causing the user to be unable to view the information collected by the probe. 4. If you want to obtain log files, you can only log in to the desktop gateway system manually, access a machine in a scene, and then manually select the desired log file for download, which is very inefficient. 5. Since the desktop gateway system needs to connect to the corresponding virtual machine or physical machine through the network, a large number of log file download operations will occupy the bandwidth resources in the scene, reduce system performance, and affect normal node management. 6. It is impossible to monitor file changes in real time, such as file size, content, permissions, checksums, etc., and it is also impossible to set certain policies, such as automatically triggering log upload and backup when the content changes abnormally. 7. Log files are not backed up. Once the network scene is closed or restarted, the virtual machine resources in it will be recycled, and the physical machine will be reset, resulting in the loss of important log files, which will affect subsequent replays. Summary of the invention

[0006] Purpose of the invention: In view of at least one of the problems existing in the above-mentioned prior art, the present invention provides a new network target range log file collection solution, deletes the desktop gateway system in the existing solution, saves resource overhead, reduces management complexity, designs a collection agent gateway / collection agent gateway system to solve the problem of network isolation, enhances probe capabilities, and reduces the pressure on the collection server.

[0007] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution:

[0008] A network range log file collection proxy gateway is equipped with at least two network cards, one of which is used to communicate with a virtual machine or a physical machine inside a range scene, and the other is used to communicate with a collection server and a file server outside the range scene; the proxy gateway is provided with an encryption module, an instruction forwarding module, a policy configuration module and a file transfer module;

[0009] The encryption module is used to encrypt and decrypt the communication between the proxy gateway and the collection server and the file server;

[0010] The instruction forwarding module is used to receive and parse the collection instructions sent by the collection server, and send the collection instructions to the probes on the specified virtual machine or physical machine according to the metadata information in the collection instructions; at the same time, the results of the execution of the instructions by each probe are collected and fed back to the collection server; if the collection instruction includes an executable script file ID, the corresponding executable script file is downloaded from the file server according to the executable script file ID, and cached in the file transfer module for download by the probe;

[0011] The policy configuration module is used to query the policy configuration information from the acquisition server according to the policy configuration ID, and save it locally in the proxy gateway for the probe to query; the policy configuration information includes the target file information and upload frequency collected by the probe;

[0012] The file transfer module is used to cache the log files collected by the probe and upload the files to the file server.

[0013] Preferably, the collection instruction includes instruction content, collection output content format, policy configuration ID, running task ID, metadata specifying which virtual machines or physical machines need to execute the collection instruction, and executable script file ID; the collection instruction execution result fed back by the probe on the virtual machine or physical machine includes the probe name, machine information where the probe is located, collection output content format, policy configuration ID, running task ID, executable script file ID, whether the command is executed successfully, error message, and collection output content.

[0014] Preferably, the encryption module refuses to respond to requests that are not from the collection server;

[0015] Preferably, the file transfer module identifies, compresses, encrypts and merges the log files uploaded by the probes.

[0016] A network range log file collection proxy gateway system, comprising a first-level collection proxy gateway and a plurality of second-level collection proxy gateways, wherein the first-level collection proxy gateway is equipped with at least two network cards, one of which is used to communicate with the second-level collection proxy gateway, and the other is used to communicate with a collection server and a file server outside a range scene; the second-level collection proxy gateway is equipped with at least two network cards, one of which is used to communicate with the first-level collection proxy gateway, and the other is used to communicate with a virtual machine or a physical machine inside the range scene; the first-level collection proxy gateway and the second-level proxy gateway are both provided with an encryption module, an instruction forwarding module, a policy configuration module and a file transfer module;

[0017] The encryption module is used to encrypt and decrypt communications outside the range scene;

[0018] The instruction forwarding module of the first-level acquisition proxy gateway is used to receive and parse the acquisition instructions sent by the acquisition server, and forward the instructions to the second-level acquisition proxy gateway; the instruction forwarding module of the second-level acquisition proxy gateway is used to receive the acquisition instructions, and according to the metadata information in the acquisition instructions, send the acquisition instructions to the probes on the designated virtual machine or physical machine; at the same time, the results of the execution of the instructions by each probe are collected, and fed back to the acquisition server through the first-level acquisition proxy gateway; if the acquisition instruction includes an executable script file ID, then according to the executable script file ID, the corresponding executable script file is downloaded from the file server through the first-level acquisition proxy gateway, and cached in the file transfer module for the probe to download;

[0019] The policy configuration module of the first-level acquisition agent gateway is used to query the policy configuration information from the acquisition server according to the policy configuration ID and save it locally; the policy configuration module of the second-level acquisition agent gateway is used to query the policy configuration information from the first-level acquisition agent gateway according to the policy configuration ID and save it locally for probe query; the policy configuration information includes the target file information and upload frequency collected by the probe;

[0020] The file transfer module of the first-level collection agent gateway is used to cache the log files uploaded by the second-level collection agent gateway and upload the files to the file server; the file transfer module of the second-level collection agent gateway is used to cache the log files uploaded by the probe and upload the files to the first-level collection agent gateway.

[0021] A network range log file collection system includes a scenario management system, a collection server, a file server, a virtual machine and / or a physical machine in a range scenario, and the collection proxy gateway; the scenario management system is used to configure the networking form of the collection proxy gateway, the collection server and the file server address, and manage log collection tasks, configure information related to collection instructions, and issue log collection instructions through the collection server; the collection server is used to receive instructions from the scenario management system, and issue the instructions to the collection proxy gateway, obtain the log file download address after the instruction is successfully executed, and return the download address to the scenario management system; the file server is used to store the executable script files required for the probe operation on the virtual machine or physical machine, and the log files collected and uploaded by the probe.

[0022] A network range log file collection system comprises a scenario management system, a collection server, a file server, a plurality of range scenarios including virtual machines and / or physical machines, and the collection agent gateway system; the scenario management system is used to configure the networking form of the collection agent gateway, the collection server and the file server address, and manage log collection tasks, configure information related to collection instructions, and issue log collection instructions through the collection server; the collection server is used to receive instructions from the scenario management system, and issue the instructions to the collection agent gateway system, obtain the log file download address after the instruction is successfully executed, and return the download address to the scenario management system; the file server is used to store the executable script files required for the probe operation on the virtual machine or physical machine, and the log files collected and uploaded by the probe.

[0023] Preferably, the log file collection in each shooting range scene corresponds to a second collection proxy gateway in the collection proxy gateway system.

[0024] Preferably, the scenario management system stores default policy configuration information and monitors different log files for different uses of shooting range scenarios; the scenario management system supports definition of one or more collection templates, and the collection templates contain multiple collection instructions and policy configuration information.

[0025] A network range log file collection method is implemented using the network range log file collection system, the method comprising:

[0026] Configure the virtual machine or physical machine where the probe needs to be installed, the network form of the collection agent gateway, the address of the collection server and file server, and the default collection policy configuration information in the scenario management system;

[0027] After the shooting range scenario is started, the virtual machine or physical machine installs and starts the probe, and downloads the default executable script from the file server through the acquisition agent gateway or the acquisition agent gateway system;

[0028] Configure the log collection task in the scenario management system, select one or more files to be downloaded, the scenario management system generates a collection instruction and sends it to the collection server, the collection server sends the collection instruction to the collection proxy gateway or collection proxy gateway system, the collection proxy gateway or collection proxy gateway system forwards the instruction to the corresponding virtual machine or physical machine probe;

[0029] After receiving the collection instruction, the probe on the virtual machine or physical machine obtains the executable script and policy configuration information specified by the instruction through the collection agent gateway or the collection agent gateway system, collects the specified log file and uploads it to the collection agent gateway or the collection agent gateway system. After the file is uploaded, the collection instruction execution result is fed back;

[0030] The acquisition proxy gateway or the acquisition proxy gateway system processes the collected log files and then uploads them to the file server; the acquisition server returns the file download address to the scene management system; the scene management system downloads the collected log files according to the file download address;

[0031] If the policy configuration information in the collection instruction received by the probe includes a regular collection interval, the probe will periodically collect and upload the corresponding log files. Subsequently, the scene management system only needs to download the latest files directly from the file server, and there is no need to issue collection instructions through the collection server.

[0032] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0033] 1. The present invention abandons the desktop gateway system, saves resource overhead, and avoids the complexity of the scene management system pushing scene and machine connection information to the desktop gateway system; through the added file server, the log files in the shooting range scene can be uniformly managed and backed up, which can avoid users' repeated and inefficient manual operations. At the same time, it can also avoid the loss of important log files caused by scene or machine shutdown or failure, can play a backup role, and can provide data support for the review of subsequent scenes.

[0034] 2. The executable script stored in the file server of the present invention can enable the probe to complete the complex task of collecting log files, and can also be set by the script to allow the probe to complete the collection task according to the specified strategy.

[0035] 3. The present invention uses a newly added collection proxy gateway to prevent machines in the network target range scenario from being exposed to the external network and thus being disturbed by the external network environment, so that the scenario can better simulate the real network environment. It can also avoid the communication blockage between the probe and the collection server caused by the isolation of the network target range scenario from the external network.

[0036] 4. The collection proxy gateway of the present invention can perform aggregation and screening on the log files collected by the probe, which can further enhance the probe capability and reduce the pressure on the collection server.

[0037] 5. The acquisition proxy gateway of the present invention can be further networked to form an acquisition proxy gateway system, which can avoid a large number of probes in different scenarios from downloading script files and uploading log files at the same time, causing great pressure on the file server and network bandwidth.

[0038] 6. By adopting the collection system of the present invention, you can configure collection tasks on the scene management system page, batch download log files from different machines or even different scenes, and automatically collect and back up important log files and monitor file changes through policy configuration.

[0039] 7. The present invention stores and backs up log files through a file server. The same file on the same virtual machine or physical machine only needs to be uploaded once. After uploading, it is stored in the file server for a long time. Subsequent downloads are downloaded through a file server outside the scene, without occupying bandwidth resources in the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the existing network range log file collection principle.

[0041] Figure 2 Schematic diagram of the network range log file collection principle of an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the network range log file collection principle according to another embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments.

[0044] The embodiment of the present invention discloses a network range log file collection agent gateway, which is equipped with at least two network cards, one of which is used to communicate with a virtual machine or a physical machine inside a range scene, and the other is used to communicate with a collection server and a file server outside the range scene; the agent gateway is provided with an encryption module, an instruction forwarding module, a policy configuration module and a file transfer module; the encryption module is used to encrypt and decrypt the communication between the agent gateway and the collection server and the file server; the instruction forwarding module is used to receive and parse the collection instruction sent by the collection server, and according to the metadata information in the collection instruction, send the collection instruction to the probe on the designated virtual machine or physical machine; at the same time, the result of each probe executing the instruction is collected and fed back to the collection server; if the collection instruction includes an executable script file ID, the corresponding executable script file is downloaded from the file server according to the executable script file ID, and cached in the file transfer module for the probe to download; the policy configuration module is used to query the policy configuration information from the collection server according to the policy configuration ID, and save it locally in the agent gateway for the probe to query; the file transfer module is used to cache the log file collected by the probe and upload the file to the file server.

[0045] In addition, in order to expand the application of multi-scenario log file collection, an embodiment of the present invention discloses a network target range log file collection agent gateway system, including a first-level collection agent gateway and multiple second-level collection agent gateways. The first-level collection agent gateway is equipped with at least two network cards, one of which is used to communicate with the second-level collection agent gateway, and the other is used to communicate with the collection server and file server outside the target range scenario; the second-level collection agent gateway is equipped with at least two network cards, one of which is used to communicate with the first-level collection agent gateway, and the other is used to communicate with the virtual machine or physical machine inside the target range scenario; the first-level collection agent gateway and the second-level agent gateway are both provided with an encryption module, an instruction forwarding module, a policy configuration module and a file transfer module.

[0046] The encryption module is used to encrypt and decrypt communications outside the range scene; the instruction forwarding module of the first-level acquisition agent gateway is used to receive and parse the acquisition instructions sent by the acquisition server and forward the instructions to the second-level acquisition agent gateway; the instruction forwarding module of the second-level acquisition agent gateway is used to receive the acquisition instructions and send the acquisition instructions to the probes on the specified virtual machine or physical machine according to the metadata information in the acquisition instructions; at the same time, the results of the execution of the instructions by each probe are collected and fed back to the acquisition server through the first-level acquisition agent gateway; if the acquisition instruction includes an executable script file ID, the corresponding executable script file is downloaded from the file server through the first-level acquisition agent gateway according to the executable script file ID, and cached to the file transfer module for the probe to download. The policy configuration module of the first-level acquisition agent gateway is used to query the policy configuration information from the acquisition server according to the policy configuration ID and save it locally; the policy configuration module of the second-level acquisition agent gateway is used to query the policy configuration information from the first-level acquisition agent gateway according to the policy configuration ID and save it locally for the probe to query; the policy configuration information includes the target file information and upload frequency collected by the probe. The file transfer module of the first-level collection agent gateway is used to cache the log files uploaded by the second-level collection agent gateway and upload the files to the file server; the file transfer module of the second-level collection agent gateway is used to cache the log files uploaded by the probe and upload the files to the first-level collection agent gateway.

[0047] Combine the following Figure 2 and Figure 3 The construction and process of the network range log file collection system using the proxy gateway / proxy gateway system of the embodiment of the present invention are described in detail.

[0048] like Figure 2As shown in the figure, a proxy gateway service is added to the network scenario. The proxy gateway service program can run on a virtual machine and start running with the scenario. The proxy gateway has two network cards, one for network communication outside the scenario and the other for network communication inside the scenario. The proxy gateway contains an encryption module, a command forwarding module, a policy configuration module, and a file transfer module. The functions of each module are as follows:

[0049] The encryption module is responsible for encrypting and decrypting the communication between itself and the acquisition server and the file server. It also has the ability to identify external requests. If the request is not from the acquisition server, it will refuse to respond to protect the network within the scene from attacks and influences from external networks.

[0050] The instruction forwarding module is used to decompose the instructions sent from the collection server, and according to the metadata information in the instructions, it will issue the instructions to one or more specified probes. At the same time, it will collect the results of each probe executing the instructions and feed them back to the collection server. If the instruction contains the content scriptId for downloading an executable script file, the instruction forwarding module will download the script file in advance and cache it in the file transfer module for the probe to use.

[0051] Instruction content template:

[0052]

[0053]

[0054] If scriptId is configured, the corresponding executable script file is downloaded through scriptId and saved locally, and the instructions in command and the local executable script file are concatenated into a string of instructions. For example: command: " / var / ossec / bin / python{script}", the executable script file downloaded through scriptId is saved in / var / osses / bin / tools / get_host_info.py, and the final instruction is " / var / ossec / bin / python / var / osses / bin / tools / get_host_info.py".

[0055] Collection instruction execution result template:

[0056]

[0057] The collection output content output depends on command and scriptId. If the execution result is the normal collection of file size, permissions and other status information, then output only contains these contents. If the execution is a log file upload instruction, then output is the file download address returned by the file server to the probe.

[0058] The policy configuration module is used to store the policy configuration information required for the probe to execute instructions. If the proxy gateway is configured with configId in the instruction it receives, it will query the collection server for the policy configuration information and save it in the local policy configuration module. Similarly, after the probe receives the instruction, it will query the corresponding policy configuration information in the policy configuration module of the proxy gateway based on the confiId. These policy configurations define the file path, upload frequency, etc. used to control probe collection, such as uploading the test.log file in the / var / log / path every 3 minutes.

[0059] Policy configuration information template:

[0060]

[0061] The file transfer module is used to cache the log files collected by the probes. It can add tags to the log files to identify which scene they come from. It can also compress, encrypt, and merge the files (for example, merge the same type of log files on multiple virtual machines or physical machines in a scene, such as / var / log / app / app.log, into a larger log file), and upload the files to the file server after processing. The transfer module can also cooperate with the instruction forwarding module to cache the executable script files downloaded from the file server. This can avoid multiple probes in the scene executing the same executable script at the same time and downloading it from the file server at the same time, which puts pressure on the file server or occupies network bandwidth.

[0062] The scenario management system saves some default policy configurations, which can monitor different log files according to the different uses of the scenario. For example, if the main purpose of the scenario is attack and defense confrontation, the focus is on monitoring security logs. If the main purpose of the scenario is a simulated competition, the focus is on monitoring traffic logs. If the main purpose of the scenario is to simulate network stress testing, the focus is on monitoring system indicator logs. Users can also add custom monitoring and collection policies.

[0063] like Figure 3As shown, the proxy gateway can also be deployed independently from the network scenario and form a network with other gateways, that is, a proxy gateway system is adopted. For example, two second collection proxy gateways, namely proxy gateways A and B, are configured in network scenario 1 and network scenario 2, and a first collection proxy gateway, namely proxy gateway C, is deployed separately. In this way, the function of the file transfer module inside the proxy gateway C can be used to mark, compress, encrypt, merge, and other operations on the log files collected and uploaded in multiple network scenarios. If machines from multiple network scenarios need to use the same script file, the proxy gateway C can cache the script file downloaded from the file server for use by machines in multiple network scenarios, further reducing the pressure on the file server.

[0064] The probes installed in each machine in the scene must communicate with the collection server through the proxy gateway. The encryption module of the proxy gateway ensures that the communication between the probe and the collection server will be encrypted. This can not only prevent the machines in the scene from being exposed to the external network environment, but also avoid the interruption of communication between the probe and the collection server due to the isolation of the scene's own network.

[0065] In addition to the network scenario, a file server needs to be deployed to store the executable script files required for the probe to run and the log files collected and uploaded by the probe from the machines in the scenario. After the probe on each machine in the scenario is started, it needs to download the default executable script from the file server through the proxy gateway. During operation, if the instruction content received by the proxy gateway contains scriptId, the script will be downloaded from the file server in advance for subsequent probe downloads. The file server can also compress and merge the log files stored in it, and can also regularly delete expired and oversized log files to save disk space and reduce bandwidth usage when users download.

[0066] Before starting the network scenario, the user will first configure in the scenario management system which virtual machines or physical machines need to install probes, the networking form of the proxy gateway, the collection server, the address of the file server, and the default collection policy configuration information. In this way, after the machines in the scenario are started, the probe on each machine will pull the specified script from the file server through the proxy gateway, and start executing the collection task according to the specified command and policy configuration information.

[0067] If the user is downloading a file for the first time, he only needs to select one or more files, and then send the instructions containing the policy configuration to the collection server through the scene management system. The collection server sends the instructions to the corresponding proxy gateway, and the proxy gateway forwards the instructions to the probe on the corresponding machine. After receiving the instructions, the probe will download and execute the corresponding script according to the instructions and policy configuration content. The script will collect the specified files and upload them to the file transfer module of the proxy gateway. After the file transfer module processes these files, it will upload them to the file server. At the same time, the probe will also collect the basic information of the file, such as name, status, size, etc., and report it to the collection server through the proxy gateway. For example, the following information is reported

[0068]

[0069]

[0070] After the upload is completed, the probe will return the information that the command has been executed successfully, and then pass it to the collection server through the proxy gateway. The collection server finds the file download address from the command execution result and returns it to the scene management system. In this way, the user can get the download file address through the scene management system and then download the collected log file through the page.

[0071] If the command received by the probe contains policy configuration information, and the policy configuration information contains a regular collection interval, such as the repeat value is true, the probe will regularly collect and upload the corresponding log file. In this way, when the user subsequently downloads the log file through the scenario management system, he can directly download the latest file from the file server instead of issuing commands through the collection server again.

[0072] Since a network scenario may contain a large number of log files that need to be monitored and collected, the scenario management system supports the definition of one or more collection templates, which contain multiple collection instructions and policy configuration information. For example, you can specify machines of a certain type of operating system and collect certain specific types of log files, such as collecting / var / log / syslog files unique to the Linux operating system.

Claims

1. A network range log file collection proxy gateway, characterized in that, it is equipped with at least two network cards, one of which is used to communicate with virtual machines or physical machines inside the range scenario, and the other is used to communicate with the collection server and file server outside the range scenario; the proxy gateway is provided with an encryption module, an instruction forwarding module, a policy configuration module, and a file transfer module; the encryption module is used to encrypt and decrypt the communication between the proxy gateway and the collection server and the file server; the instruction forwarding module is used to receive and parse the collection instructions sent by the collection server, and according to the metadata information in the collection instructions, send the collection instructions to the probes on the specified virtual machines or physical machines; at the same time, collect the results of the instruction execution by each probe and feedback them to the collection server; if the collection instruction includes an executable script file ID, download the corresponding executable script file from the file server according to the executable script file ID and cache it in the file transfer module for the probe to download; the policy configuration module is used to query the policy configuration information from the collection server according to the policy configuration ID and save it locally in the proxy gateway for the probe to query; the policy configuration information includes the target file information collected by the probe and the upload frequency; the file transfer module is used to cache the log files collected by the probe and upload the files to the file server.

2. The network range log file collection proxy gateway according to claim 1, characterized in that, the collection instruction includes instruction content, collection output content format, policy configuration ID, running task ID, metadata including which virtual machines or physical machines need to execute the collection instruction, and executable script file ID; the collection instruction execution result feedback by the probe on the virtual machine or physical machine includes probe name, probe location machine information, collection output content format, policy configuration ID, running task ID, executable script file ID, whether the command is executed successfully, error information, and collection output content.

3. The network range log file collection proxy gateway according to claim 1, characterized in that, for requests not from the collection server, the encryption module refuses to respond.

4. The network range log file collection proxy gateway according to claim 1, characterized in that, the file transfer module performs identification, compression, encryption, and merging processing on the log files uploaded by the probe.

5. A network range log file collection proxy gateway system, characterized in that, it includes a first-level collection proxy gateway and multiple second-level collection proxy gateways. The first-level collection proxy gateway is equipped with at least two network cards, one of which is used to communicate with the second-level collection proxy gateway, and the other is used to communicate with the collection server and file server outside the range scenario; the second-level collection proxy gateway is equipped with at least two network cards, one of which is used to communicate with the first-level collection proxy gateway, and the other is used to communicate with virtual machines or physical machines inside the range scenario; both the first-level collection proxy gateway and the second-level proxy gateway are provided with an encryption module, an instruction forwarding module, a policy configuration module, and a file transfer module; The encryption module is used for encrypting and decrypting the communication outside the range scenario; The instruction forwarding module of the first-level acquisition proxy gateway is used for receiving and parsing the acquisition instructions sent by the acquisition server and forwarding the instructions to the second-level acquisition proxy gateway; the instruction forwarding module of the second-level acquisition proxy gateway is used for receiving the acquisition instructions, and according to the metadata information in the acquisition instructions, sending the acquisition instructions to the probes on the specified virtual machine or physical machine; meanwhile, collecting the results of the probes executing the instructions and feeding them back to the acquisition server through the first-level acquisition proxy gateway; if the acquisition instructions include the ID of the executable script file, then according to the ID of the executable script file, downloading the corresponding executable script file from the file server through the first-level acquisition proxy gateway and caching it in the file transfer module for the probes to download; The policy configuration module of the first-level acquisition proxy gateway is used for querying the policy configuration information from the acquisition server according to the policy configuration ID and saving it locally; The policy configuration module of the second-level acquisition proxy gateway is used for querying the policy configuration information from the first-level acquisition proxy gateway according to the policy configuration ID and saving it locally for the probes to query; the policy configuration information includes the target file information collected by the probes and the upload frequency; The file transfer module of the first-level acquisition proxy gateway is used for caching the log files uploaded by the second-level acquisition proxy gateway and uploading the files to the file server; the file transfer module of the second-level acquisition proxy gateway is used for caching the log files uploaded by the probes and uploading the files to the first-level acquisition proxy gateway.

6. A network range log file acquisition system, characterized in that, it includes a scenario management system, an acquisition server, a file server, virtual machines and / or physical machines in the range scenario, and the acquisition proxy gateway according to claim 1; the scenario management system is used for configuring the networking form of the acquisition proxy gateway, the addresses of the acquisition server and the file server, and managing the log acquisition task, configuring the information related to the acquisition instructions, and issuing the log acquisition instructions through the acquisition server; The acquisition server is used for receiving the instructions of the scenario management system, issuing the instructions to the acquisition proxy gateway, obtaining the log file download address after the instructions are successfully executed, and returning the download address to the scenario management system; The file server is used for storing the executable script files required for the probes to run on the virtual machines or physical machines, and the log files collected and uploaded by the probes.

7. A network range log file acquisition system, characterized in that, it includes a scenario management system, an acquisition server, a file server, multiple range scenarios including virtual machines and / or physical machines, and the acquisition proxy gateway system according to claim 4; The scenario management system is used for configuring the networking form of the acquisition proxy gateway, the addresses of the acquisition server and the file server, and managing the log acquisition task, configuring the information related to the acquisition instructions, and issuing the log acquisition instructions through the acquisition server; The collection server is used to receive instructions from the scenario management system, send the instructions to the collection proxy gateway system, obtain the log file download address after the instructions are successfully executed, and return the download address to the scenario management system; The file server is used to store the executable script files required for the probe to run on the virtual machine or physical machine, as well as the log files collected and uploaded by the probe.

8. The network range log file collection system according to claim 7, characterized in that, The collection of log files in each range scenario corresponds to a second collection proxy gateway in the collection proxy gateway system.

9. The network range log file collection system according to claim 6 or 7, characterized in that, The scenario management system stores default policy configuration information and monitors different log files for different uses of the range scenario; the scenario management system supports defining one or more collection templates, and the collection templates include multiple collection instructions and policy configuration information.

10. A method for collecting network range log files, characterized in that, It is implemented by using the network range log file collection system according to claim 6 or 7, and the method includes: Configure the virtual machine or physical machine on which the probe needs to be installed in the scenario management system, the networking form of the collection proxy gateway, the addresses of the collection server and the file server, and the default collection policy configuration information; After the range scenario is started, the virtual machine or physical machine installs and starts the probe, and downloads the default executable script from the file server through the collection proxy gateway or the collection proxy gateway system; Configure the log collection task in the scenario management system, select one or more files to be downloaded, the scenario management system generates a collection instruction and sends it to the collection server, the collection server sends the collection instruction to the collection proxy gateway or the collection proxy gateway system, and the collection proxy gateway or the collection proxy gateway system forwards the instruction to the probe on the corresponding virtual machine or physical machine; After the probe on the virtual machine or physical machine receives the collection instruction, it obtains the executable script and policy configuration information specified by the instruction through the collection proxy gateway or the collection proxy gateway system, collects the specified log file and uploads it to the collection proxy gateway or the collection proxy gateway system. After the file upload is completed, it feeds back the execution result of the collection instruction; The collection proxy gateway or the collection proxy gateway system processes the collected log files and then uploads them to the file server; the collection server returns the file download address to the scenario management system; the scenario management system downloads the collected log files according to the file download address; If the policy configuration information in the collection instruction received by the probe includes a regular collection interval, the probe regularly collects the corresponding log files and uploads them. Subsequently, only the latest files need to be directly downloaded from the file server in the scenario management system, and there is no need to send a collection instruction through the collection server.

Images