Detection engine cluster detection method and system based on ETCD

Abstract

The invention relates to a detection engine cluster detection method based on ETCD, and the method comprises the steps: building a database table based on a preset building rule; based on a preset configuration rule, determining a corresponding relationship between the detection engine service monitor and the to-be-monitored node; acquiring node information uploaded by each detection engine service monitor, and adding the node information to a database table according to the correspondence between the detection engine service monitors and the to-be-monitored nodes on the basis of a preset storage rule; and based on a preset processing rule, according to the node information, judging whether a node corresponding to the node information is inactivated: if so, deleting the node and the node information corresponding to the node. The method has the effect that the service deployment and management mode of the detection engine is more convenient.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a detection engine cluster detection method and system based on ETCD. Background Art

[0002] The network environment detection system relies on multiple detection engine services distributed in the cyberspace environment. For example, the data in search engines such as Baidu and Google all come from their data collection engines distributed around the world. In the early stages of the network environment detection system, the number of detection engine services deployed was small, and manual deployment and operation and maintenance could maintain the normal operation of the system.

[0003] However, with the development of network environment, the business volume has increased. As more and more detection engine services need to be deployed, the disadvantages of relying on manual maintenance of detection engine services have gradually emerged. Manual maintenance is prone to errors on the one hand, and it also consumes a lot of manpower and easily causes inconvenience on the other hand. Summary of the invention

[0004] The first purpose of this application is to provide a detection engine cluster detection method based on ETCD, which has the characteristic of making the detection engine service deployment and management more convenient.

[0005] The above-mentioned application objective 1 of the present application is achieved through the following technical solutions: A detection engine cluster detection method based on ETCD includes establishing a database table based on preset establishment rules; Based on the preset configuration rules, determine the corresponding relationship between the detection engine service listener and the node to be monitored; Obtain the node information uploaded by each detection engine service listener, and add the node information to the database table according to the corresponding relationship between the detection engine service listener and the node to be monitored based on the preset saving rules; Based on the preset processing rules, judging whether the node corresponding to the node information is inactivated according to the node information: If so, delete the node and the node information corresponding to the node.

[0006] By adopting the above technical solution, a database table is established to store the node information of the detection engine. Each detection engine node is provided with a corresponding listener, which monitors the detection engine node in real time so as to timely update the data in the database table to ensure subsequent use and maintenance. When the database table is updated, the node information of the detection engine node is added to the database table. If the data output by the listener shows that the detection engine node is inactivated, the corresponding data in the database table is deleted to ensure the timeliness of the data in the database table. The detection engine nodes are managed by automatically monitoring the detection engine nodes, which saves manpower and makes the management of the detection engine nodes more convenient.

[0007] Optionally, the method for establishing a database table based on preset information collection rules includes: Set node data and running data as column names respectively; Select the engine node ID in the node data and define it as the primary key; Get the database table.

[0008] By adopting the above technical solution, two kinds of data are included in the database table. One is the configuration information of the node itself, namely the node data, and the other is the operation data of the detection engine node monitored by the listener, so that the status of the detection engine node can be understood and judged by viewing the data in the database table later. The engine node ID is set as the primary key, and there will be only one piece of relevant data for each detection engine node in the database table to reduce the occurrence of duplicate data and improve storage efficiency.

[0009] Optionally, the method for determining the correspondence between the detection engine service listener and the node to be monitored based on a preset configuration rule includes: Register the detection engine service listener; Associate the detection engine service listener with the node to be monitored.

[0010] By adopting the above technical solution, each detection engine node is provided with a unique listener to monitor it, so as to monitor the detection engine node and transmit the node data and operation data of the detection engine node to the corresponding module.

[0011] Optionally, the method of obtaining the node information uploaded by each detection engine service listener and adding the node information to the database table according to the corresponding relationship between the detection engine service listener and the node to be monitored based on a preset saving rule includes: Get the node information of the detection monitoring node; Based on the preset judgment rules and node information, determine whether the detection engine node is a newly registered node: If yes, save the node information to the database table.

[0012] By adopting the above technical solution, when receiving node information, it is first determined whether the detection engine node is a newly registered node. If it is a newly registered node, it indicates that there is no relevant information in the database table. If it is not a newly registered node, it means that the node information of the detection engine already exists in the database table, and the corresponding operation data part in the database table is directly updated.

[0013] Optionally, the method for judging whether the detection engine node is a newly registered node based on the node information based on a preset judgment rule includes: Extract the engine node ID; Traverse the latest database and compare the engine node ID; If there is no identical detection engine node ID, the detection node is a newly registered node.

[0014] By adopting the above technical solution, the method for determining whether the detection engine node is a newly registered node is: querying the engine node ID in the database table to see whether there is a completely identical engine node ID. If there is no identical engine node ID, it means that the node is a newly registered node, thereby reducing duplicate data in the database table and reducing the waste of storage space.

[0015] Optionally, the method for determining whether a node corresponding to the node information is inactivated based on the preset processing rule and according to the node information includes: Acquire detection engine operation data, wherein the detection engine operation data includes CPU operation status and CPU operation ratio; According to the preset receiving time, if the detection engine operation data is not received within the preset receiving time, the detection engine node is marked as an inactive node.

[0016] By adopting the above technical solution, if the operation data is not received within the expected time, it means that the operation status of the detection engine node is abnormal. The engine node is marked as an inactive node in time, and relevant operations are performed to remind the user to perform relevant processing.

[0017] Optionally, also include: Save the node information of the inactive nodes and mark them as abnormal data.

[0018] By adopting the above technical solution, the data contained in the node information of the deactivated node is saved, and the deactivated data output by the system is associated with the node information of the deactivated node and marked as abnormal data for later maintenance by the user.

[0019] The second application objective of the present application is achieved through the following technical solutions: A detection engine cluster detection system based on ETCD, comprising: A data acquisition module, used to acquire node information of the detection engine node according to preset information collection rules; The data processing module is used to regularly delete the inactive node information data and update the database according to the preset processing rules; The data recording module is used to record abnormal data based on the preset recording rules and the inactivated node information data.

[0020] By adopting the above technical solution, the data acquisition module collects the node information of the detection engine and hands the received node information to the data processing module for relevant processing. Waste data will appear during the data processing process. The data recording module collects and records the waste data for later user viewing or maintenance.

[0021] In summary, the present application includes at least one of the following beneficial technical effects: 1. Establish a database table to store the node information of the detection engine. Each detection engine node is equipped with a corresponding listener, which monitors the detection engine node in real time to update the data in the database table in time to ensure later use and maintenance. When updating the database table, add the node information of the detection engine node to the database table. If the data output by the listener shows that the detection engine node is inactive, delete the corresponding data in the database table to ensure the timeliness of the data in the database table. By automatically monitoring the detection engine node, the detection engine node is managed, which saves manpower and makes the management of the detection engine node more convenient. ; 2. Save the data contained in the node information of the deactivated node, associate the deactivated data output by the system with the node information of the deactivated node, and mark it as abnormal data for the user's later maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a system block diagram of the ETCD-based detection engine cluster detection system provided in this application.

[0023] Figure 2 It is a flowchart of the detection engine cluster detection method based on ETCD provided in this application.

[0024] Figure 3 It is a structural schematic diagram of the electronic device provided by this application.

[0025] In the figure, 100, data acquisition module; 200, data processing module; 300, data recording module; 401, CPU; 402, ROM; 403, RAM; 404, I / O interface; 405, input part; 406, output part; 407, storage part; 408, communication part; 409, drive; 410, removable medium. DETAILED DESCRIPTION

[0026] The following is combined with Figure 1-3 This application is described in further detail.

[0027] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the following will be combined with the appended Figure 1-3 , the technical solutions in the embodiments of the present application are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0029] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0030] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0031] The embodiment of the present application provides a detection engine cluster detection method based on ETCD, and the main process of the method is described as follows.

[0032] like Figure 1 As shown: Step S1: Create a database table based on preset creation rules.

[0033] Specifically, the relevant description information of the detection engine node to be collected is set first, and a corresponding database table is established, so that the database can be used to store the node information of the detection engine node in the later stage, and the detection engine node can be managed in an informationized manner. In this embodiment, the node information of the detection engine node includes node data and operation data. The node data of the detection engine node includes: engine node ID, node name, node custom label, node deployment area code, detection engine service task type code, detection engine service execution strategy, extended field, etc. The operation data of the node information of the detection engine node includes: CPU operation ratio, CPU memory ratio, etc. In the database table, the engine node ID, node name, node custom label, node deployment area code, detection engine service task type code, detection engine service execution strategy, and extended field are all set as column names, and the engine node ID is selected as the primary key of this database table, so as to complete the establishment of the database table. Later, the node information of the detection engine node in this database table is modified according to the status of the detection engine node, so as to realize the information management of data, reduce labor costs, and make the detection engine service deployment and management method more convenient.

[0034] After creating the database table, proceed to the next step to configure the listener.

[0035] Step S2: Based on the preset configuration rules, determine the corresponding relationship between the detection engine service listener and the node to be monitored.

[0036] Specifically, in the network environment detection system, each detection engine node is registered with a detection engine service listener, and the detection engine node is associated with the detection engine service listener. After the detection engine service listener is registered, it monitors the status of the detection engine node by monitoring the service port, and can detect whether the corresponding detection engine node is operating normally, so as to facilitate the subsequent information management of the detection engine node.

[0037] After setting up the detection environment, proceed to the next step to deploy the detection engine node.

[0038] Step S3: Obtain the node information uploaded by each detection engine service listener, and based on a preset saving rule, add the node information to the database table according to the corresponding relationship between the detection engine service listener and the node to be monitored.

[0039] Specifically, the network environment detection system includes an Agent module for sending service requests and an ETCD registration service center for processing request information. Each time a detection engine node is set up, the node data of the detection engine node must be manually associated with the detection engine node, and then the detection engine service is started. The detection engine service listener monitors the running status of the detection engine in real time, and sends the node information of the detection engine to the ETCD registration service center through the Agent module. After receiving the registration request, the ETCD registration service center selects the detection engine node ID in the node information, and traverses the information stored in the database table, and compares the detection engine node ID with the detection engine node ID stored in the database table one by one. If there is no exactly the same detection engine node ID, the node information under the detection engine node ID is saved to the database table and the database table is updated.

[0040] On the other hand, if it is found during the comparison process that the database table has a completely identical detection engine node ID, the operation data of the detection engine node transmitted by the Agent module is saved to the corresponding position in the database, and the database table is updated.

[0041] When setting up the detection engine node, the relevant description information of the detection engine node is manually configured. Later, the information of the detection engine node is automatically transmitted, obtained and saved in the database table by the module in the system. If relevant data needs to be retrieved during management, the node information under a detection engine node ID can be retrieved from the latest database. This saves a lot of manpower and makes the management of the detection engine service more convenient.

[0042] In order to monitor the running status of the detection engine node in real time, proceed to the next step.

[0043] Step S4: Based on the preset processing rules and according to the node information, determine whether the node corresponding to the node information is inactivated: if so, delete the node and the node information corresponding to the node.

[0044] Specifically, a unified preset receiving time is set in the network detection engine system configuration strategy. This receiving time is the time interval for each detection engine node to send operating data. If the ETCD registration service center does not receive the operating data of a detection engine node transmitted by the Agent module within the preset receiving time, the detection engine node is inactive. The ETCD registration service center traverses the database table, deletes the detection engine nodes that have not received the operating data within the time limit from the database table, and updates the database table to obtain the latest database, so as to achieve the purpose of always storing valid information in the latest database.

[0045] The detection engine nodes that are operating normally are recorded in the latest database for retrieval by other devices or viewing by users, but abnormal information also needs to be processed and proceed to the next step.

[0046] Step S5: Save the node information of the deactivated node and mark it as abnormal data.

[0047] Specifically, obtain the operation log in the database table and traverse the operation log. When it is found that the ETCD registration service center has deleted the node information due to a reception timeout, record the detection engine node ID, retrieve the node information under the detection engine node ID in the backup data, and associate the running status of the detection engine node with the corresponding position, where the running status of the detection engine node should at least include: the status of the detection engine node, the date of deactivation of the detection engine node, the time of deactivation of the detection engine node, etc.

[0048] The above recorded data is marked as abnormal data, which can be used by users to review or repair the node information of the abnormal detection engine node, thereby facilitating the user's maintenance of the network detection engine system.

[0049] The present application also provides a detection engine cluster detection system based on ETCD, referring to Figure 2 ,include: The data acquisition module 100 is used to acquire node information data of the detection engine node according to a preset information collection rule; The data processing module 200 is used to regularly delete the inactive node information data and update the database according to the preset processing rules; The data recording module 300 is used to record abnormal data based on the preset recording rules and the deactivated node information data.

[0050] Specifically, the above modules for acquiring, processing and recording data are well known to persons skilled in the art and will not be elaborated here.

[0051] The present application embodiment discloses an electronic device. Figure 3 The electronic device includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 407 to the random access memory (RAM) 404. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, the ROM 402 and the RAM 403 are connected to each other through a bus. The input / output (I / O) interface 404 is also connected to the bus.

[0052] The following components are connected to the I / O interface 404: an input section 405 including a keyboard, a mouse, etc.; an output section 406 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 407 including a hard disk, etc.; and a communication section 408 including a network interface card such as a LAN card, a modem, etc. The communication section 408 performs communication processing via a network such as the Internet. A drive 409 is also connected to the I / O interface 404 as needed. A removable medium 410, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 409 as needed, so that a computer program read therefrom is installed into the storage section 407 as needed.

[0053] In particular, according to an embodiment of the present application, the above reference flow chart Figure 1 The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 408, and / or installed from a removable medium 410. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the device of the present application are executed.

[0054] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0055] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned application concept. For example, the above features are replaced with (but not limited to) technical features with similar functions applied in the present application.

Claims

1. A detection engine cluster detection method based on ETCD, characterized in that: include, Create a database table based on the preset creation rules; Based on the preset configuration rules, determine the corresponding relationship between the detection engine service listener and the node to be monitored; Obtain the node information uploaded by each detection engine service listener, and add the node information to the database table according to the corresponding relationship between the detection engine service listener and the node to be monitored based on the preset saving rules; Based on the preset processing rules and according to the node information, it is determined whether the node corresponding to the node information is inactivated: If so, delete the node and the node information corresponding to the node.

2. According to the ETCD-based detection engine cluster detection method of claim 1, it is characterized in that: The method for establishing a database table based on preset information collection rules includes: Set node data and running data as column names respectively; Select the engine node ID in the node data and define it as the primary key; Get the database table.

3. The detection engine cluster detection method based on ETCD according to claim 1, characterized in that: The method for determining the corresponding relationship between the detection engine service listener and the node to be monitored based on the preset configuration rule includes: Register the detection engine service listener; Associate the detection engine service listener with the node to be monitored.

4. The detection engine cluster detection method based on ETCD according to claim 2, characterized in that: The method of obtaining the node information uploaded by each detection engine service listener and adding the node information to the database table according to the corresponding relationship between the detection engine service listener and the node to be monitored based on the preset saving rule includes: Get the node information of the detection monitoring node; Based on the preset judgment rules and node information, determine whether the detection engine node is a newly registered node: If yes, save the node information to the database table.

5. The detection engine cluster detection method based on ETCD according to claim 1, characterized in that: The method for judging whether the detection engine node is a newly registered node based on the preset judgment rule and the node information includes: Extract the engine node ID; Traverse the latest database and compare the engine node ID; If there is no identical detection engine node ID, the detection node is a newly registered node.

6. The detection engine cluster detection method based on ETCD according to claim 2, characterized in that: The method for determining whether a node corresponding to the node information is inactivated based on the preset processing rule and according to the node information includes: Acquire detection engine operation data, wherein the detection engine operation data includes CPU operation status and CPU operation ratio; According to the preset receiving time length, if the detection engine operation data is not received within the preset receiving time length, the detection engine node is marked as an inactive node.

7. The detection engine cluster detection method based on ETCD according to claim 4, characterized in that: Also includes: Save the node information of the inactive nodes and mark them as abnormal data.

8. A detection engine cluster detection system based on ETCD, characterized in that: include: A data acquisition module (100), used to acquire node information of a detection engine node according to a preset information collection rule; A data processing module (200) is used to regularly delete inactive node information data and update the database according to a preset processing rule; The data recording module (300) is used to record abnormal data based on the preset recording rules and the deactivated node information data.

9. An intelligent terminal, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program which can be loaded by the processor and executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 7.

Images