Method, system and equipment for synchronizing high-availability configuration file of firewall and medium
By using the CRUD callback method of etcd database and ligato framework in the high availability profile synchronization of firewall, full and incremental synchronization is achieved, solving the problems of manual operations and third-party tools in the existing technology, and improving efficiency and automation level , ensuring real-time synchronization and security of configuration files.
Patent Information
- Application Number
- CN202411145395.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-12-06
AI Technical Summary
The existing method of high availability profile synchronization for firewalls requires administrators to operate manually or use third-party tools, which has limitations and risks, is inefficient and not automated enough.
By starting the web file server for full synchronization, using the etcd database to monitor file changes and calling the make-mirror method for incremental synchronization, the routine synchronization between the CRUD function and the configuration file synchronization routine is cancelled, and the callback method is used to implement the configuration file Incremental synchronization.
The purpose of notifying the backup host configuration file changes is achieved, network communication overhead is reduced, synchronization performance is optimized, automation and real-time is improved, and the possibility of human error is reduced.
Smart Images

Figure CN119094537A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of configuration file synchronization, and in particular to a firewall high-availability configuration file synchronization method, system, device and medium. Background Art
[0002] The high availability mode (HA) of a firewall is an important mechanism for protecting network security. As the first line of defense for network security, a firewall can monitor, filter, and control the data flow in and out of the network. The high availability mode is a way to ensure that the firewall can still operate normally in the face of failures or large amounts of traffic.
[0003] In the past, firewalls were usually deployed in a single-machine manner, which means that all network traffic passes through a single firewall device. However, this single-point failure design can easily lead to network disconnection, increased latency, and downtime, seriously affecting business continuity and user experience.
[0004] To solve this problem, high availability mode was introduced into the firewall architecture. Its basic idea is to achieve load balancing and redundant backup by clustering multiple firewall devices. In this way, when one device fails or is unavailable, other devices will take over traffic processing to ensure the continuity and security of network traffic.
[0005] High availability mode can be active-standby mode or active-active mode. In active-standby mode, one device acts as the primary device to handle all traffic, while the other devices are in standby mode. When the primary device fails, the standby device automatically takes over. In active mode, all devices are in working state and jointly handle traffic to achieve load balancing.
[0006] In high availability mode, the standby machine needs to synchronize the file configuration of the host machine. Traditional synchronization configuration methods include manual synchronization, SCP protocol transmission, RCP protocol replication, FTP file transfer, TFTP file transfer, etc.
[0007] Manual synchronization: The administrator manually copies the configuration files from the primary device to the standby device.
[0008] SCP, Secure Copy: Use the SCP protocol to copy and transfer files. The administrator can package the configuration files on the primary device and then use SCP to transfer the files between the primary and backup devices.
[0009] RCP, Remote Copy: Use the RCP protocol to copy files.
[0010] FTP, File Transfer Protocol: Use the FTP protocol for file transfer. The administrator can upload the configuration file to the FTP server and then download the file from the FTP server on the backup device.
[0011] TFTP, Trivial File Transfer Protocol: Use the TFTP protocol for file transfer. The administrator can set up a TFTP server on the primary device, and the backup device downloads the configuration file from the TFTP server.
[0012] Manual synchronization: This requires coordination and operation by the administrator, which may introduce human errors and delays and is not efficient enough.
[0013] SCP, Secure Copy: This method is relatively simple, but requires manual operation.
[0014] RCP, Remote Copy: Similar to SCP, RCP can also transfer files between primary and backup devices. However, RCP is relatively weak in terms of security, so additional security measures may be required.
[0015] FTP, File Transfer Protocol: This method requires additional FTP server and configuration.
[0016] TFTP, Trivial File Transfer Protocol: TFTP is a simple file transfer protocol suitable for small file transfers.
[0017] These traditional configuration file synchronization methods basically require administrators to perform manual operations or use third-party tools, which have certain limitations and risks. Summary of the invention
[0018] Aiming at the problem that the existing configuration file synchronization method requires manual operation by an administrator or the use of a third-party tool, and has certain limitations and risks, the present invention proposes a firewall high-availability configuration file synchronization method, system, device and medium; firstly, a web file server is started according to a heartbeat IP, and a high-availability standby machine is called to fully synchronize the configuration file to be synchronized; secondly, according to a file change monitoring routine of a high-availability host, a changed file of a monitoring cycle is written into an etcd database; and a make-mirror method is called to pull a list of configuration files to be synchronized from the etcd database; then, a high-availability standby machine is called to publish a database transaction, and a map mapping is obtained according to a UC operation of the database transaction; finally, the map mapping is monitored, and the configuration files to be synchronized are incrementally synchronized; a callback method is adopted to realize the incremental synchronization of the configuration file, and the routine synchronization between the CRUD function and the configuration file synchronization routine is cancelled, so as to optimize the synchronization performance.
[0019] The specific implementation contents of the present invention are as follows: A method for synchronizing a firewall high-availability configuration file, specifically comprising the following steps: Step S1: Start the web file server according to the obtained heartbeat IP, and call the high-availability standby machine to fully synchronize the configuration files to be synchronized; Step S2: According to the file change monitoring routine of the high-availability host, write the changed file of a monitoring cycle to the etcd database; Step S3: Call the make-mirror method to pull the list of configuration files to be synchronized from the etcd database; Step S4: calling the high-availability standby machine to publish a database transaction, and obtaining a map mapping according to the UC operation of the database transaction; Step S5: Call the high-availability standby file synchronization routine to monitor the map mapping and incrementally synchronize the configuration files to be synchronized.
[0020] In order to better implement the present invention, further, the step S4 specifically includes the following steps: Step S41: The high-availability standby machine publishes a database transaction; Step S42: Calling the UC operation according to the ligato agent, obtaining the map[string]int mapping corresponding to the configuration file to be synchronized according to the UC operation; wherein string represents the file path + file name, int=0 indicates that the configuration file has not changed and does not need to be synchronized, and int>0 indicates that the configuration file has changed and needs to be synchronized.
[0021] In order to better implement the present invention, further, step S5 specifically includes the following steps: Step S51: obtaining the state number lastrevision of the previous user-defined identifier key, and obtaining the state number nextrevision of the current user-defined identifier key; Step S52: compare the state number lastrevision and the state number nextrevision. If the state number nextrevision is greater than the state number lastrevision, the value under the current user-defined identifier key is changed. Step S53: call the mirror maker method to pull the list of configuration files to be synchronized of the high-availability host from the etcd database, step S54: call the high-availability standby machine to publish the database transaction, and obtain the map[string]int mapping corresponding to the configuration files to be synchronized according to the UC operation and the list of configuration files to be synchronized; Step S55: According to the map[string]int mapping, the synchronization routine monitors the map[string]int mapping and incrementally synchronizes the configuration files to be synchronized.
[0022] In order to better implement the present invention, further, the specific operation of step S53 is: publish an empty transaction of the etcd database, and check the process of the etcd database transaction according to the empty transaction of the etcd database; if the revision value of the etcd database is not 0, call the mirror maker method to pull the list of configuration files to be synchronized of the high-availability host from the etcd database.
[0023] In order to better implement the present invention, further, before performing step S1, the following steps are included: Compile the configuration file to be synchronized into the filesynclist.json file according to the path + name format and string.
[0024] In order to better implement the present invention, further, the database transaction includes an update transaction and a create transaction.
[0025] In order to better implement the present invention, further, the synchronization control plane of the high availability mode of the configuration file to be synchronized is vpp-agent.
[0026] Based on the above-proposed firewall high-availability configuration file synchronization method, in order to better implement the present invention, further, a firewall high-availability configuration file synchronization system is proposed, including a full synchronization unit and an incremental synchronization unit; The full synchronization unit is used to start the web file server and call the high-availability standby machine to fully synchronize the configuration files to be synchronized; The incremental synchronization unit is used to first write the changed files of a monitoring cycle to the etcd database according to the file change monitoring routine of the high-availability host; secondly, call the make-mirror method to pull the list of configuration files to be synchronized from the etcd database; then call the high-availability standby machine to publish the database transaction, and obtain the map mapping according to the UC operation of the database transaction; finally, call the high-availability standby machine file synchronization routine to monitor the map mapping, and incrementally synchronize the configuration files to be synchronized.
[0027] Based on the above-proposed method for synchronizing a high-availability configuration file of a firewall, in order to better implement the present invention, further, an electronic device is proposed, comprising a memory and a processor; the memory is used to store a computer program; when the computer program is executed on the processor, the above-proposed method for synchronizing a high-availability configuration file of a firewall is implemented.
[0028] Based on the above-mentioned firewall high-availability configuration file synchronization method, in order to better implement the present invention, further, a computer-readable storage medium is proposed, on which computer instructions are stored; when the computer instructions are executed on the above-mentioned electronic device, the above-mentioned firewall high-availability configuration file synchronization method is implemented.
[0029] The present invention has the following beneficial effects: (1) The present invention realizes the purpose of notifying the backup machine of changes in the configuration file of its host machine based on the mirroring method of the etcd database and the CRUD callback method triggered by the ligato framework for etcd key value monitoring, and reduces the overhead of self-built sockets for network communication.
[0030] (2) The present invention determines whether the configuration file has been changed by monitoring the timestamp of the host configuration file. Instead of adopting a periodic polling method, a callback method is adopted to realize the incremental synchronization of the corresponding configuration file, thereby canceling the routine synchronization between the CRUD function and the configuration file synchronization routine, thereby achieving the effect of optimizing the synchronization performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 A schematic diagram of a flow chart of a firewall high-availability configuration file synchronization method provided by the present invention. DETAILED DESCRIPTION
[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore should not be regarded as limiting the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without making creative work are within the scope of protection of the present invention.
[0033] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0034] The complete English corresponding to the professional terms and English abbreviations involved in this embodiment are as follows: HA stands for High Available, which means high availability.
[0035] SCP stands for Secure Copy, which is a secure copy protocol based on the SSH protocol.
[0036] RCP stands for Remote Copy, which is the remote copy protocol.
[0037] FTP stands for File Transfer Protocol.
[0038] TFTP stands for Trivial File Transfer Protocol, which is a small file transfer protocol.
[0039] Embodiment 1: This embodiment provides a method for synchronizing a firewall high-availability configuration file, which specifically includes the following steps: Step S1: Start the web file server according to the obtained heartbeat IP, and call the high-availability standby machine to fully synchronize the configuration files to be synchronized.
[0040] Step S2: According to the file change monitoring routine of the high-availability host, the changed file of a monitoring cycle is written to the etcd database.
[0041] Step S3: Call the make-mirror method to pull the list of configuration files to be synchronized from the etcd database.
[0042] Step S4: calling the high-availability standby machine to publish a database transaction, and obtaining a map mapping according to the UC operation of the database transaction.
[0043] The step S4 specifically comprises the following steps: Step S41: The high-availability standby machine publishes a database transaction; Step S42: Calling the UC operation according to the ligato agent, obtaining the map[string]int mapping corresponding to the configuration file to be synchronized according to the UC operation; wherein string represents the file path + file name, int=0 indicates that the configuration file has not changed and does not need to be synchronized, and int>0 indicates that the configuration file has changed and needs to be synchronized.
[0044] Step S5: Call the high-availability standby file synchronization routine to monitor the map mapping and incrementally synchronize the configuration files to be synchronized.
[0045] In order to better implement the present invention, further, step S5 specifically includes the following steps: Step S51: obtaining the state number lastrevision of the previous user-defined identifier key, and obtaining the state number nextrevision of the current user-defined identifier key; Step S52: compare the state number lastrevision and the state number nextrevision. If the state number nextrevision is greater than the state number lastrevision, the value under the current user-defined identifier key is changed. Step S53: calling the mirror maker method to pull the list of configuration files to be synchronized of the high-availability host from the etcd database; Furthermore, the specific operation of step S53 is: publishing an empty transaction of the etcd database, and checking the process of the etcd database transaction according to the empty transaction of the etcd database; if the revision value of the etcd database is not 0, calling the mirror maker method to pull the list of configuration files to be synchronized of the high-availability host from the etcd database.
[0046] Step S54: calling the high-availability standby machine to publish a database transaction, and obtaining a map[string]int mapping corresponding to the configuration file to be synchronized according to the UC operation and the list of configuration files to be synchronized; Step S55: According to the map[string]int mapping, the synchronization routine monitors the map[string]int mapping and incrementally synchronizes the configuration files to be synchronized.
[0047] Furthermore, before performing step S1, the following steps are included: Compile the configuration file to be synchronized into the filesynclist.json file according to the path + name format and string.
[0048] Furthermore, the database transaction includes an update transaction and a create transaction.
[0049] Furthermore, the synchronization control plane of the high availability mode of the configuration file to be synchronized is vpp-agent.
[0050] Working principle: This embodiment first starts the web file server according to the heartbeat IP, and calls the high-availability standby machine to fully synchronize the configuration files to be synchronized; secondly, according to the file change monitoring routine of the high-availability host, writes the changed file of a monitoring cycle into the etcd database; and calls the make-mirror method to pull the list of configuration files to be synchronized from the etcd database; then calls the high-availability standby machine to publish a database transaction, and obtains the map mapping according to the UC operation of the database transaction; finally, monitors the map mapping, and incrementally synchronizes the configuration files to be synchronized; the callback method is used to realize the incremental synchronization of the configuration files, cancels the routine synchronization between the CRUD function and the configuration file synchronization routine, and optimizes the synchronization performance.
[0051] Embodiment 2: This embodiment is based on the above embodiment 1. Figure 1 As shown, a specific embodiment is described in detail.
[0052] This embodiment is implemented on the basis of the ligato framework and the database etcd, using the make-mirror method of the etcd database and the CRUD callback method of the kv transaction of the ligato framework. The synchronization process is as follows: Mirror maker can mirror the cluster in a prefix in the etcd database to another etcd database. This mirroring process is very efficient. Most of the delay comes from network delay. Applying this technology in HA mode eliminates network delay.
[0053] This article uses the make mirror technology based on the official recommended usage of the etcd database; however, we have improved this method and enhanced the robustness of the program.
[0054] Synchronization is divided into two steps. First, full synchronization is performed. After the full synchronization is completed, the incremental synchronization phase begins.
[0055] Incremental synchronization will record the revision of the previous key, called lastrevison, and the revision of the current key, called nextrevision. If nextrevision>lastrevison, the value under the key has changed. At this time, before using mirror maker to pull data from the other end, you need to check whether the transaction execution of etcd on this end is normal. Simply publish an empty etcd transaction to check the correctness of the etcd transaction process execution. If no error is returned, mirror the data.
[0056] During the mirror maker process, the database revision of the mirror target will be recorded. During the incremental synchronization process, revision==0 will be checked. If it is equal to 0, the program will panic and exit. However, since the etcd revision value of the mirror target is 0 in some error situations, such as abnormal restart of the mirror target, or initialization, or initialization is completed without any revision operation, in this case, the mirror target error information will be collected first, then the mirror target etcd will be restarted, and finally the mirror will be made again.
[0057] When each incremental synchronization is completed, you should also check whether the revision of the currently stored target image is the same as the revision of the target image. If they are different, you should update the revision. Otherwise, it will cause the incremental synchronization to fail and even cause the agent core to crash.
[0058] This embodiment uses vpp-agent as the control plane for configuration file synchronization in high-availability mode, and integrates file detection, web file transfer server, and web client into the agent for unified management. The agent provides etcd transaction detection, and provides the files that need to be synchronized according to the transaction, ultimately achieving the effect of adaptive file synchronization.
[0059] A web file server will be enabled on both the high-availability master and standby machines.
[0060] After the master-slave negotiation is completed, the standby machine performs a full synchronization of the files that need to be synchronized.
[0061] After the full synchronization is completed, the incremental synchronization of configuration files begins.
[0062] This embodiment proposes a configuration file synchronization method in a firewall high availability mode, which aims to synchronize the configuration files of the host in real time, safely and without omission. The specific method is as follows: Step 1: Write the files to be synchronized into the filesynclist.json file in the form of path + name.
[0063] Step 2: Both the master and the slave start the web file server for configuration file transmission and security verification; the web username and password verification is used to ensure security, and the heartbeat IP is used to enable the web service, which has a certain degree of service concealment; the etcd mirroring method and the ligato framework triggering the CRUD callback method of etcd key value monitoring are used to notify the slave of changes in the master's configuration file. This also reduces the cost of self-built sockets for network communication.
[0064] Step 3: The standby machine starts the file synchronization routine and performs a full synchronization of the configuration files after the first negotiation is completed.
[0065] Step 4: After the full synchronization of the standby machine is completed, enter the incremental synchronization phase.
[0066] Step 5: The host enables the configuration file change monitoring routine and writes the changed files involved in a monitoring cycle to the etcd database; the timestamp of the host configuration file is monitored to determine whether the configuration file has changed.
[0067] Step 6: The standby machine uses the mirror method to pull the list of configuration files that need to be synchronized from the host etcd database.
[0068] Step 7: The standby etcd publishes the database update / create transaction, and the ligato agent calls the relevant UC operation. In the UC operation, it finds the mapping of map[string]int corresponding to the file to be synchronized, where string represents the file path + file name, int=0 means that the configuration file has not changed and does not need to be synchronized, and int>0 means that the configuration file has changed and needs to be synchronized.
[0069] Step 8: The standby file synchronization routine monitors the map mapping and synchronizes the incremental files that need to be synchronized. Instead of using the periodic polling method, the callback method is used to achieve incremental synchronization of the corresponding configuration files. This cancels the routine synchronization between the CRUD function and the configuration file synchronization routine, achieving the effect of optimizing synchronization performance.
[0070] Working principle: This embodiment solves the technical problem of configuration file synchronization brought about by the existing firewall high-availability mode deployment. From the perspective of automation, real-time, security, and robustness, it provides a solution for synchronizing configuration files in the high-availability mode of firewall configuration through etcd database mirror technology, ligato framework kv scheduler technology, and file transfer web server technology.
[0071] The other parts of this embodiment are the same as those of the above-mentioned embodiment 1, and thus will not be described in detail.
[0072] Embodiment 3: This embodiment, based on any one of the above-mentioned embodiments 1-2, proposes a firewall high-availability configuration file synchronization system; including a full synchronization unit and an incremental synchronization unit; The full synchronization unit is used to start the web file server and call the high-availability standby machine to fully synchronize the configuration files to be synchronized; The incremental synchronization unit is used to first write the changed files of a monitoring cycle to the etcd database according to the file change monitoring routine of the high-availability host; secondly, call the make-mirror method to pull the list of configuration files to be synchronized from the etcd database; then call the high-availability standby machine to publish the database transaction, and obtain the map mapping according to the UC operation of the database transaction; finally, call the high-availability standby machine file synchronization routine to monitor the map mapping, and incrementally synchronize the configuration files to be synchronized.
[0073] This embodiment also proposes an electronic device, including a memory and a processor; the memory is used to store a computer program; when the computer program is executed on the processor, the above-mentioned firewall high-availability configuration file synchronization method is implemented.
[0074] This embodiment further provides a computer-readable storage medium, on which computer instructions are stored; when the computer instructions are executed on the above-mentioned electronic device, the above-mentioned firewall high-availability configuration file synchronization method is implemented.
[0075] The other parts of this embodiment are the same as any one of the above-mentioned embodiments 1-2, so they will not be repeated here.
[0076] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.
Claims
1. A method for synchronizing a firewall high-availability configuration file, characterized in that: The specific steps include: Step S1: Start the web file server according to the obtained heartbeat IP, and call the high-availability standby machine to fully synchronize the configuration files to be synchronized; Step S2: According to the file change monitoring routine of the high-availability host, write the changed file of a monitoring cycle to the etcd database; Step S3: Call the make-mirror method to pull the list of configuration files to be synchronized from the etcd database; Step S4: calling the high-availability standby machine to publish a database transaction, and obtaining a map mapping according to the UC operation of the database transaction; Step S5: Call the high-availability standby file synchronization routine to monitor the map mapping and incrementally synchronize the configuration files to be synchronized.
2. A method for synchronizing a firewall high-availability configuration file according to claim 1, characterized in that: The step S4 specifically comprises the following steps: Step S41: The high-availability standby machine publishes a database transaction; Step S42: Calling the UC operation according to the ligato agent, obtaining the map[string]int mapping corresponding to the configuration file to be synchronized according to the UC operation; wherein string represents the file path + file name, int=0 indicates that the configuration file has not changed and does not need to be synchronized, and int>0 indicates that the configuration file has changed and needs to be synchronized.
3. A method for synchronizing a firewall high-availability configuration file according to claim 2, characterized in that: The step S5 specifically comprises the following steps: Step S51: obtaining the state number lastrevision of the previous user-defined identifier key, and obtaining the state number nextrevision of the current user-defined identifier key; Step S52: compare the state number lastrevision and the state number nextrevision. If the state number nextrevision is greater than the state number lastrevision, the value under the current user-defined identifier key is changed. Step S53: call the mirror maker method to pull the list of configuration files to be synchronized of the high-availability host from the etcd database, step S54: call the high-availability standby machine to publish the database transaction, and obtain the map[string]int mapping corresponding to the configuration files to be synchronized according to the UC operation and the list of configuration files to be synchronized; Step S55: According to the map[string]int mapping, the synchronization routine monitors the map[string]int mapping and incrementally synchronizes the configuration files to be synchronized.
4. A method for synchronizing a firewall high-availability configuration file according to claim 3, characterized in that: The specific operation of step S53 is: publishing an empty transaction of the etcd database, and checking the process of the etcd database transaction according to the empty transaction of the etcd database; if the revision value of the etcd database is not 0, calling the mirror maker method to pull the list of configuration files to be synchronized of the high-availability host from the etcd database.
5. A method for synchronizing a firewall high-availability configuration file according to claim 1, characterized in that: Before performing step S1, the method includes: Compile the configuration file to be synchronized into the filesynclist.json file according to the path + name format and string.
6. A method for synchronizing a firewall high-availability configuration file according to claim 1, characterized in that: The database transactions include update transactions and create transactions.
7. The method for synchronizing a firewall high-availability configuration file according to any one of claims 1 to 6, characterized in that: The synchronization control plane of the high availability mode of the configuration file to be synchronized is vpp-agent.
8. A firewall high-availability configuration file synchronization system, characterized in that: Including full synchronization unit and incremental synchronization unit; The full synchronization unit is used to start the web file server and call the high-availability standby machine to fully synchronize the configuration files to be synchronized; The incremental synchronization unit is used to first write the changed files of a monitoring cycle to the etcd database according to the file change monitoring routine of the high-availability host; secondly, call the make-mirror method to pull the list of configuration files to be synchronized from the etcd database; then call the high-availability standby machine to publish the database transaction, and obtain the map mapping according to the UC operation of the database transaction; finally, call the high-availability standby machine file synchronization routine to monitor the map mapping, and incrementally synchronize the configuration files to be synchronized.
9. An electronic device, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; when the computer program is executed on the processor, the firewall high-availability configuration file synchronization method as described in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions; when the computer instructions are executed on the electronic device as described in claim 9, the firewall high-availability configuration file synchronization method as described in any one of claims 1-7 is implemented.
