Data backup method, device and equipment
By using lock-free ring queue and reverse indexing technology during data backup, the problems of large overhead and low efficiency of data transmission are solved, and efficient and low-impact data backup and transmission are achieved.
Patent Information
- Application Number
- CN202411713555.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-27
- Publication Date
- 2025-03-07
AI Technical Summary
In the prior art, data transmission overhead is high and inefficient during data backup, resulting in the data traffic carried by the backup link being close to 30% of the normal forwarded data, which is very large.
Efficient transmission is achieved by writing the storage address of the target data to a lock-free ring queue and deleting redundant data to be sent but has died using reverse index.
It avoids the context switching overhead caused by using mutex locks, improves the synchronization performance of multi-threads, reduces data packaging time and backup packet space, and improves transmission efficiency and performance.
Smart Images
Figure CN119576608A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information security technology, and in particular to a data backup method, device and equipment. Background Art
[0002] A stateful firewall is a firewall technology that can track the status of network connections. By using the Virtual Router Redundancy Protocol (VRRP), two firewalls are combined into a virtual security gateway device to achieve communication reliability. At the same time, a dedicated heartbeat line is required between the two active and standby firewalls to back up configuration information, link status information (i.e., session data), database information, etc. in real time. However, the data traffic carried by the backup link is close to 30% of the normal forwarded data, which is very large. In the prior art, backing up a large amount of data is costly and the efficiency of backing up data is low. Therefore, how to efficiently and effectively transmit backup data becomes an important task. Summary of the invention
[0003] The embodiments of the present application provide a data backup method, device and equipment to solve the problem of high data transmission overhead and low efficiency during data backup.
[0004] In a first aspect, an embodiment of the present application provides a data backup method, comprising:
[0005] When it is determined that N processes generate target data, for each process, control the process to write the storage address of the target data into a first target position of the lock-free circular queue, and send the target index position of the first target position in the lock-free circular queue to the process, so that the process saves the target index position in the target data; wherein the first target position is the position pointed to by the producer head index of the ring buffer, and N is a positive integer greater than or equal to 1;
[0006] When it is determined that the target data of the target process needs to be deleted, a target index position is determined from the target data of the target process, and a storage address of the target data of the target process is deleted from the lock-free circular queue based on the target index position; the target process is any one of the N processes;
[0007] When it is determined that the target data needs to be sent, the target data to be sent is acquired based on the storage address in the lock-free circular queue, and the target data to be sent is sent to the backup device.
[0008] In a possible implementation, controlling the process to write the storage address of the target data into the first target position of the lock-free circular queue includes:
[0009] Control the process to obtain a producer head index in the ring buffer and a storage object in a first target position of the lock-free ring queue pointed to by the producer head index;
[0010] When the process determines that the storage object in the position pointed to by the producer head index in the lock-free circular queue is the same as the storage object in the first target position obtained, the process is controlled to write the storage address of the target data into the first target position of the lock-free circular queue.
[0011] In a possible implementation, the method further includes:
[0012] Setting the index position of the preset producer's next index to the next index position of the producer's head index;
[0013] After the storage address of the target data is written into the first target position, the index position of the producer head index in the ring buffer is updated to the index position of the preset producer next index.
[0014] In a possible implementation, the method further includes:
[0015] When the process determines that the storage object in the position pointed to by the producer header index in the lock-free circular queue is different from the storage object in the acquired first target position, the process is controlled to re-acquire the producer header index in the circular buffer.
[0016] In a possible implementation, deleting the storage address of the target data of the target process from the lock-free circular queue based on the target index position includes:
[0017] Control the target process to obtain the consumer head index and the producer tail index in the ring buffer;
[0018] When it is determined that the target index position of the target process is between the consumer head index and the producer tail index, the data in the target index position in the lock-free circular queue is set to invalid, and a message indicating that the data in the target index position has been successfully deleted is returned to the target process.
[0019] In a possible implementation, the method further includes:
[0020] Obtaining a storage object at a second target location pointed to by the consumer head index;
[0021] Before returning a message to the target process indicating successful deletion of the data in the target index position, it is determined that the storage object in the position pointed to by the consumer head index in the lock-free circular queue is the same as the acquired storage object of the second target position.
[0022] In a possible implementation, the method further includes:
[0023] When it is determined that the storage object at the position pointed to by the consumer head index in the lock-free circular queue is different from the storage object at the second target position obtained, and the target index position is not between the consumer head index and the producer tail index, a message is returned to the target process to express that the data deletion in the target index position has failed.
[0024] In a possible implementation, acquiring the target data to be sent based on the storage address in the lock-free circular queue includes:
[0025] Obtaining the consumer head index in the ring buffer and the number of nodes to be obtained;
[0026] Determine the index position of the next index of the preset consumer based on the number of nodes to be acquired and the consumer head index;
[0027] For each index position between the consumer head index and the preset consumer next index, obtain the storage address in the third target position pointed to by each index position in the lock-free circular queue, and obtain the target data to be sent based on the storage address in the third target position.
[0028] In a second aspect, an embodiment of the present application provides a data backup device, including:
[0029] A first processing unit is used for, when determining that N processes generate target data, for each process, controlling the process to write the storage address of the target data into a first target position of the lock-free circular queue, and sending the target index position of the first target position in the lock-free circular queue to the process, so that the process saves the target index position in the target data; wherein the first target position is a position pointed to by a producer head index of the ring buffer, and N is a positive integer greater than or equal to 1;
[0030] The first processing unit is further configured to determine a target index position from the target data of the target process when it is determined that the target data of the target process needs to be deleted, and delete a storage address of the target data of the target process from the lock-free circular queue based on the target index position; the target process is any one of the N processes;
[0031] The second processing unit, when determining that the target data needs to be sent, obtains the target data to be sent based on the storage address in the lock-free circular queue, and sends the target data to be sent to the backup device.
[0032] In a possible implementation, the first processing unit, when controlling the process to write the storage address of the target data into the first target position of the lock-free circular queue, is specifically configured to:
[0033] Control the process to obtain a producer head index in the ring buffer and a storage object in a first target position of the lock-free ring queue pointed to by the producer head index;
[0034] When the process determines that the storage object in the position pointed to by the producer head index in the lock-free circular queue is the same as the storage object in the first target position obtained, the process is controlled to write the storage address of the target data into the first target position of the lock-free circular queue.
[0035] In a possible implementation manner, the first processing unit is further configured to:
[0036] Setting the index position of the preset producer's next index to the next index position of the producer's head index;
[0037] After the storage address of the target data is written into the first target position, the index position of the producer head index in the ring buffer is updated to the index position of the preset producer next index.
[0038] In a possible implementation manner, the first processing unit is further configured to:
[0039] When the process determines that the storage object in the position pointed to by the producer header index in the lock-free circular queue is different from the storage object in the acquired first target position, the process is controlled to re-acquire the producer header index in the circular buffer.
[0040] In a possible implementation, the first processing unit, when deleting the storage address of the target data of the target process from the lock-free circular queue based on the target index position, is specifically configured to:
[0041] Control the target process to obtain the consumer head index and the producer tail index in the ring buffer;
[0042] When it is determined that the target index position of the target process is between the consumer head index and the producer tail index, the data in the target index position in the lock-free circular queue is set to invalid, and a message indicating that the data in the target index position has been successfully deleted is returned to the target process.
[0043] In a possible implementation manner, the first processing unit is further configured to:
[0044] Obtaining a storage object at a second target location pointed to by the consumer head index;
[0045] Before returning a message to the target process indicating successful deletion of the data in the target index position, it is determined that the storage object in the position pointed to by the consumer head index in the lock-free circular queue is the same as the acquired storage object of the second target position.
[0046] In a possible implementation manner, the first processing unit is further configured to:
[0047] When it is determined that the storage object at the position pointed to by the consumer head index in the lock-free circular queue is different from the storage object at the second target position obtained, and the target index position is not between the consumer head index and the producer tail index, a message is returned to the target process to express that the data deletion in the target index position has failed.
[0048] In a possible implementation, when the second processing unit obtains the target data to be sent based on the storage address in the lock-free circular queue, it is specifically configured to:
[0049] Obtaining the consumer head index in the ring buffer and the number of nodes to be obtained;
[0050] Determine the index position of the next index of the preset consumer based on the number of nodes to be acquired and the consumer head index;
[0051] For each index position between the consumer head index and the preset consumer next index, obtain the storage address in the third target position pointed to by each index position in the lock-free circular queue, and obtain the target data to be sent based on the storage address in the third target position.
[0052] In a third aspect, an embodiment of the present application provides an execution device, including:
[0053] A memory for storing program instructions;
[0054] The processor is used to obtain program instructions stored in the memory and execute the method described in the first aspect and different implementation methods of the first aspect according to the obtained program instructions.
[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes computer instructions. When the computer instructions are executed by a computer, the method described in the first aspect and different implementation methods of the first aspect are implemented.
[0056] In a fifth aspect, the present application provides a computer program product, comprising: a computer program code, when the computer program code runs on a computer, enables the computer to execute the method described in the first aspect and different implementation methods of the first aspect.
[0057] The effects of this application are as follows:
[0058] In this application, the target data is stored in a lock-free circular queue, avoiding the context switching overhead caused by the use of mutex locks, achieving high-performance synchronization of the state data to be sent by multiple business threads, and minimizing the performance impact on the forwarding plane. In addition, after the storage address of the state data to be sent is written into the lock-free circular queue, the index position is saved in the state data. Through the reverse index, redundant state data that is in the state to be sent but has disappeared can be deleted, thereby reducing the time for data packaging and the space for backup messages, and improving transmission efficiency and performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0060] Figure 1 A schematic diagram of a data backup scenario provided in an embodiment of the present application;
[0061] Figure 2 A schematic diagram of a data backup scenario provided in an embodiment of the present application;
[0062] Figure 3 A schematic diagram of adding target data to a lock-free circular queue provided in an embodiment of the present application;
[0063] Figure 4 A schematic diagram of adding target data to a lock-free circular queue provided in an embodiment of the present application;
[0064] Figure 5 A schematic diagram of a lock-free circular queue before writing when there is contention provided by an embodiment of the present application;
[0065] Figure 6A schematic diagram of refreshing the pointer of core2 when there is contention provided in an embodiment of the present application;
[0066] Figure 7 A schematic diagram of refreshing core2 to write target data into a lock-free circular queue when there is contention provided in an embodiment of the present application;
[0067] Figure 8 A schematic diagram of deleting target data is provided for an embodiment of the present application;
[0068] Fig. 9 A schematic diagram of deleting target data when there is competition provided by an embodiment of the present application;
[0069] Fig.10 Another schematic diagram of deleting target data when there is competition provided by an embodiment of the present application;
[0070] Fig.11 A schematic diagram of obtaining target data from a lock-free circular queue provided in an embodiment of the present application;
[0071] Fig.12 A schematic diagram of an extended VRRP message format provided in an embodiment of the present application;
[0072] Fig.13 A schematic diagram of message packaging provided in an embodiment of the present application;
[0073] Fig.14 A schematic diagram of data transmission during data backup provided in an embodiment of the present application;
[0074] Fig.15 A schematic diagram of a data backup device provided in an embodiment of the present application;
[0075] Fig.16 A schematic diagram of the structure of an execution device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0076] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0077] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.
[0078] In order to better understand the data backup method, device and equipment proposed in this application, the technical terms involved in this application are explained below.
[0079] Virtual Router Redundancy Protocol (VRRP): By combining multiple routing devices into a virtual routing device and allowing the physical routers to hide their own information, when one of the devices fails, data communication can be switched to other devices, thus achieving reliable communication.
[0080] Lock-free circular queue: It is a special circular "first-in, first-out" queue. It uses the CPU's compare and swap (CAS, a type of atomic operation that can be used to implement uninterrupted data exchange operations in multi-threaded programming) feature and the "producer and consumer model" to achieve multi-threaded lock-free reading and writing, avoiding the context switching overhead caused by frequent locking, and achieving high-performance reading and writing.
[0081] Each lock-free circular queue data structure contains two pairs of pointers: head and tail, representing the head and tail. One pair is for the producer (represented by prod in this application), and the other pair is for the consumer (represented by cons in this article). The following article uses the producer head index prod_head, the producer tail index prod_tail, the consumer head index cons_head, and the consumer tail index cons_tail to represent these four pointers respectively.
[0082] Table 1 Variable name definition for queue entry and exit operations
[0083] Serial number Variable Name meaning 1 g_prod_head Producer head index in the ring buffer 2 g_prod_tail The producer's tail index in the ring buffer 3 g_cons_head Index of the consumer head in the ring buffer 4 g_cons_tail The consumer's tail index in the ring buffer 5 l_prod_head Producer header index in temporary variable 6 l_prod_tail The producer's tail index in the temporary variable 7 l_cons_head Index of consumer header in temporary variable 8 l_cons_tail Producer header index in temporary variable 9 l_prod_next The next index of the producer in the temporary variable 10 l_cons_next The next index of the consumer in the temporary variable
[0084] Each CPU has an exclusive cache, and the temporary variable is the value stored in the local cache of the corresponding CPU that has not yet been updated to the global ring queue. The variables of the ring buffer represent the shared ring queue values in the storage.
[0085] Stateful firewall: also known as dynamic packet filtering firewall, is a firewall technology that can track the status of network connections. When a connection is established, the firewall will record the relevant information of the connection (referred to as session information in this article), such as source address, destination address, port number, protocol type, etc. During the connection process, the firewall will use this information to determine whether the subsequent data packets belong to this connection. If so, it will be allowed to pass; if not, it will be rejected.
[0086] Compare And Swap (CAS) instruction: contains three parameters, namely the memory address A of the shared variable, the value B used for comparison, and the new value C of the shared variable. The value at the memory address A can be updated to the new value C only when the value at the memory address A is equal to B. Therefore, the CAS instruction can guarantee atomicity.
[0087] When firewalls are deployed at the egress of an enterprise network, ensuring uninterrupted network transmission becomes an important task. By using the VRRP protocol, two firewalls are combined into a virtual security gateway device to achieve communication reliability. At the same time, a dedicated heartbeat line is required between the two active and standby firewalls to back up configuration information, link status information, database information, etc. in real time. According to industry experience, the data traffic carried by the backup link is close to 30% of the normal forwarding data, which is very large. How to efficiently and high-performance transmit backup data becomes an important task.
[0088] In response to the above problems, the embodiments of the present application provide a data backup method, device and medium, which store the target data in a lock-free circular queue, avoid the context switching overhead caused by the use of mutex locks, and achieve high-performance synchronization of the state data to be sent by multiple business threads, thereby minimizing the performance impact on the forwarding plane. In addition, after the storage address of the target data to be sent is written into the lock-free circular queue, the index position is saved in the state data. Through the reverse index, redundant target data that is in the state to be sent but has disappeared can be deleted, thereby reducing the time for data packaging and the space for backup messages, and improving transmission efficiency and performance.
[0089] like Figure 1 As shown, Figure 1The schematic diagram of data backup involved in the present application is exemplarily shown, which includes a first device 100 and a second device 200. The first device 100 includes a first gateway 101, and the second device includes a second gateway 201. The first device 100 is connected to a local area network LAN, and the second device 200 is connected to a wireless local area network WLAN. The data generated by the first device 100 is sent to the second device 200 through the first gateway 101 and the first virtual firewall 102. The second device 200 is sent to the first device 100 through the second gateway 201 and the second virtual firewall 202. The first virtual firewall 102 and the second virtual firewall 202 are two virtual firewalls that back up each other, connected by a dedicated heartbeat line, for transmitting connection status data and configuration data, etc.
[0090] See also Figure 2 FIG. 1 is a flowchart of a data backup method provided by an embodiment of the present application. The method can be performed by Figure 1 The first device 100 can be implemented, and the second device 200 can also be implemented, which is not specifically limited in this application. The specific process of the water backup method provided in this application is as follows:
[0091] S201, when it is determined that N processes generate target data, for each process, the control process writes the storage address of the target data into the first target position of the lock-free circular queue, and sends the target index position of the first target position in the lock-free circular queue to the process, so that the process saves the target index position in the target data.
[0092] The first target position is the position pointed to by the producer head index of the ring buffer, N is a positive integer greater than or equal to 1, and the target data may be configuration data or connection status data.
[0093] In some embodiments, the control process writes the storage address of the target data to the first target position of the lock-free circular queue, which can be specifically implemented in the following manner: the control process obtains the producer header index in the circular buffer, and the storage object in the first target position of the lock-free circular queue pointed to by the producer header index. Further, when the process determines that the storage object in the position pointed to by the producer header index in the lock-free circular queue is the same as the storage object in the obtained first target position, the control process writes the storage address of the target data to the first target position of the lock-free circular queue.
[0094] In some embodiments, the index position of the preset producer next index may be set to the next index position of the producer header index. After the storage address of the target data is written to the first target position, the index position of the producer header index in the ring buffer is updated to the index position of the preset producer next index.
[0095] As an example, different processes run on different cores. For example, if any one of the N processes runs on core1, Figure 3 For the convenience of description, Figure 3 The lock-free circular queue is represented in the form of a horizontal queue, where the next node of the last node in the horizontal queue is the first node in the horizontal queue. In some scenarios, the producer head index g_prod_head in the ring buffer can be sent to the temporary variable l_prod_head in core1, and the content of the position pointed to by the producer head index g_prod_head in the lock-free ring queue is sent to core1 as the storage object of l_cons_tail. Furthermore, the l_prod_next pointer can be pointed to the next object of the object pointed to by the producer head index g_prod_head in the lock-free ring queue. If there is not enough space in the ring buffer, an error is returned. In some scenarios, the consumer tail index g_cons_tail can also be sent to the temporary variable l_cons_tail in core1. Since the head index position is constantly changing during storage, when the head index and the tail index are the same, it means that the ring queue is full and there are no free nodes.
[0096] In some embodiments, when the storage object in the position pointed to by the ring buffer g_prod_head in the lock-free ring queue is determined by comparing and exchanging instructions through a CAS instruction, and is the same as the storage object of the temporary variable l_prod_head, the storage address of the target data generated in core1 (represented by obj4) is written to the position pointed to by g_prod_head in the lock-free ring queue. Further, g_prod_head of the ring buffer is moved to the position of the temporary variable l_prod_next, such as Figure 4 shown.
[0097] In some scenarios, when writing the storage address of the target data to the location pointed to by g_prod_head in the lock-free ring queue, g_prod_tail in the ring buffer is updated to point to g_prod_head of the ring buffer, such as Figure 4 Further, the index position of the written storage address in the lock-free circular queue is recorded and sent to core1, so that core1 stores the index position in the target data.
[0098] In some embodiments, when the process determines that the storage object in the position pointed to by the producer header index in the lock-free circular queue is different from the storage object in the acquired first target position, the control process re-acquires the producer header index in the circular buffer. Further, based on whether the storage object in the position pointed to by the re-acquired producer header index in the lock-free circular queue is the same as the storage object in the position pointed to by the producer header index in the lock-free circular queue, the target data may be stored in the position pointed to by the re-acquired producer header index in the lock-free circular queue if they are the same.
[0099] Continuing with the above example, let's take two processes running on core1 and core2 respectively. Figure 5 As shown, when two processes need to write the storage address of the target data into the lock-free circular queue, the producer head index g_prod_head and the consumer tail index g_cons_tail are sent to the temporary variables l_prod_head and l_cons_tail in core1, and the temporary variables l_prod_head and l_cons_tail in core2, respectively. And the content of the position of g_prod_head in the lock-free circular queue is used as the storage object of the temporary variable l_prod_head in core1 and the storage object of the temporary variable l_prod_head in core2, respectively.
[0100] Furthermore, when core1 has written the storage address of the target data (obj4) into the lock-free circular queue, g_prod_head in the circular buffer points to the temporary variable l_prod_next in core1, but the temporary variables l_prod_head and l_cons_tail in core2 have not changed. Because the storage object of the temporary variable l_prod_head in core2 was empty before, but obj4 was stored in this position in the lock-free circular queue, the two are different. At this time, core2 is controlled to re-acquire the producer head index g_prod_head and consumer tail index g_cons_tail in the circular buffer to refresh the temporary variables l_prod_head and l_prod_next in core2, pointing to the current g_prod_head and the next node of g_prod_head in the circular buffer, respectively, as shown in FIG. Figure 6 shown.
[0101] Furthermore, core2 finds that the ring buffer g_prod_head has not been changed through a compare-and-swap instruction (CAS), moves the ring buffer g_prod_head to the position of the temporary variable l_prod_next, and then adds the storage address of the target data (represented by obj5) to the lock-free ring queue, as shown in Figure 7 Then, the g_prod_tail of the ring buffer can be updated to point to the g_prod_head of the ring buffer, and the position index of the object obj5 written can be recorded and sent to core2.
[0102] S202, when it is determined that the target data of the target process needs to be deleted, determine the target index position from the target data of the target process, and delete the storage address of the target data of the target process from the lock-free circular queue based on the target index position.
[0103] The target process is any one of the N processes.
[0104] In some embodiments, the target process may be controlled to obtain the consumer head index and the producer tail index in the ring buffer. When it is determined that the target index position of the target process is between the consumer head index and the producer tail index, the data in the target index position in the lock-free ring queue is set to invalid, and a message indicating that the data in the target index position has been successfully deleted is returned to the target process.
[0105] In some embodiments, the storage object of the second target location pointed to by the consumer head index can be obtained; before returning a message to the target process indicating that the data in the target index location has been successfully deleted, it is determined that the storage object in the location pointed to by the consumer head index in the lock-free circular queue is the same as the obtained storage object of the second target location.
[0106] As an example, Figure 8 As shown, the target data to be deleted is the target data of core1, which can be represented by obj3 in the lock-free circular queue. First, the g_cons_head and g_prod_tail of the circular buffer can be stored in the temporary variables l_cons_head and l_prod_tail of core1. And the temporary variable consumer next index l_cons_next points to l_cons_head.
[0107] In some scenarios, if the index position (obj3) of the connection number to be deleted is between l_cons_head and l_prod_tail, it is determined whether the storage object at the position pointed to by the temporary variable l_cons_head in the lock-free circular queue is the same as the storage object at the position pointed to by the ring buffer g_cons_head in the lock-free circular queue. If they are the same, the value of obj3 in the lock-free circular queue is set to invalid, and a message of successful deletion is returned. During the deletion of obj3, the values of the index pointers do not change.
[0108] In other scenarios, if core2 consumes a storage object in a position in the lock-free circular queue, causing the g_cons_head and g_cons_tail of the ring buffer to point to obj2. At this time, through the CAS instruction, it is found that the storage object at the position pointed to by the temporary variable l_cons_head in the lock-free circular queue is different from the storage object at the position pointed to by the g_cons_head of the ring buffer in the lock-free circular queue, but the object obj3 to be deleted is still in the valid area. At this time, the value of obj3 in the lock-free circular queue can be set to invalid, and a message of successful deletion is returned, such as Fig. 9 shown.
[0109] In some embodiments, when it is determined that the storage object at the location pointed to by the consumer head index in the lock-free circular queue is different from the storage object acquired at the second target location, and the target index location is not between the consumer head index and the producer tail index, a message is returned to the target process indicating that the data deletion at the target index location has failed.
[0110] As an example, the target data to be deleted is the target data of core1, which can be represented by obj3 in the lock-free circular queue. First, the g_cons_head and g_prod_tail of the circular buffer can be stored in the temporary variables l_cons_head and l_prod_tail of core1. And the temporary variable consumer next index l_cons_next points to l_cons_head.
[0111] If core2 consumes the storage objects in the three positions in the lock-free circular queue, causing the g_cons_head and g_cons_tail of the ring buffer to point to obj4, core1 compares the CAS instruction and finds that the storage object at the position pointed to by the temporary variable l_cons_head in the lock-free circular queue is different from the storage object at the position pointed to by g_cons_head of the ring buffer in the lock-free circular queue, and obj3 to be deleted is no longer in the valid area, such as Fig.10As shown, a message indicating that the deletion failed is returned. It can be considered that the target data has been packaged and sent, and the upper-layer logic needs to add the message for deleting the target data to the lock-free circular queue so that the backup device deletes the target data after receiving the message for deleting the target data.
[0112] S203, when it is determined that the target data needs to be sent, the target data to be sent is obtained based on the storage address in the lock-free circular queue, and the target data to be sent is sent to the backup device.
[0113] In some embodiments, a consumer head index and a target number to be obtained in a ring buffer may be obtained; based on the target number to be obtained and the consumer head index, an index position of a preset consumer next index may be determined. For each index position between the consumer head index and the preset consumer next index, a storage address in a third target position pointed to by each index position in the lock-free ring queue is obtained, and target data to be sent is obtained based on the storage address in the third target position.
[0114] As an example, take the process sending the target data running on core3, the number of targets obtained by core3 is 3, and the current g_cons_head of the ring buffer points to obj1. The g_cons_head and g_prod_tail of the ring buffer are stored in the temporary variables l_cons_head and l_prod_tail in core3. Furthermore, the temporary variable l_cons_next can be offset according to the number of targets obtained, 3, to point to obj4, such as Fig.11 As shown. Further, the CAS instruction is then used to determine whether g_cons_head has been modified due to use by other processes. If it has not been modified, the storage address in each position between l_cons_head and l_cons_next is obtained, and the corresponding target data is obtained based on the storage address. Then, g_cons_head is pointed to obj4. After the modification is successful, g_cons_tail also points to obj4.
[0115] In some embodiments, if it is found that the storage address corresponding to obj3 is set to invalid when acquiring data, the packing of obj3 is skipped in the data packing stage, thereby saving packing time and improving packing efficiency.
[0116] In this application, the target data to be sent is packaged and sent through the VRRP protocol. In this application, the VRRP protocol is extended, and the extended VRRP message format is as follows: Fig.12 shown.
[0117] The changes to the VRRP protocol are shown in the following table:
[0118]
[0119] The Type field is expanded and type 6 is added to indicate backup data.
[0120] The Priority field is used to carry the type number of the backup data and to distinguish data types, such as configuration data, session data, and security configuration file data.
[0121] The CountIPvX Addrs field is used to carry the number of backup data entries, indicating the number of complete backup data entries.
[0122] The rsvd and Adver Int fields are combined to carry the length of a single backup data record.
[0123] The IPvX Address field is used to store the backed-up data content.
[0124] The extended VRRP protocol retains VRID information and can distinguish multiple data sources. VRRP can be directly included in IP messages, with a simple message structure and a 16-byte message header, which saves space and makes the message transmission more efficient.
[0125] As an example, after obtaining the target data to be sent from the lock-free ring queue, the Type field in the VRRP message header can be set to 6 based on the target data, indicating that the target data to be sent is backup data, and the backup data type number, the number of backup data items field, and the length field of a single backup data item are determined based on the target data to be sent, thereby obtaining a data packet. Further, the address of the backup device and the address of the device to be backed up are added to the IP header to process routing and forwarding at the network layer. Further, the physical address of the device to be backed up and the physical address of the backup device are added to the MAC header to process physical transmission at the data link layer, thereby obtaining a complete backup data message, such as Fig.13 Through the above packaging method, compared with the prior art packaging method in which the IP header is followed by the VGMP (12 bytes) and HRP (36 bytes) headers, and then the DATA data is packaged, the packaging method of the present application improves the packaging efficiency.
[0126] In this application, the data flow diagram is as follows Fig.14As shown in the figure, the target data to be backed up is session data, the firewall that generates the data to be backed up is called the main firewall, and the firewall that receives the data to be backed up is called the backup firewall. During the operation, the service thread in the main firewall generates session data. The address of the session data is written to the lock-free circular queue in real time through a shallow copy. The lock-free circular queue returns index information to the service thread, and the service thread records the index in the session data.
[0127] In some scenarios, session data disappears while it is waiting in the queue to be sent. Especially after the system is restarted, when the session data needs to be backed up in batches, it is necessary to wait until the batch backup is completed before the data in the lock-free circular queue can be sent in real time. In this case, the session data will wait in the queue for a long time. Due to the end of the business, the session data will disappear. In some embodiments, when it is necessary to delete session data (i.e., the disappeared session data) from the lock-free circular queue, since the index of the session data in the lock-free circular queue is saved in the session data when the address of the session data is stored in the lock-free circular queue, the index in the session data can be used to delete the address information of the session data of the corresponding node from the lock-free circular queue and set it to "invalid".
[0128] In some embodiments, the corresponding data in the nodes of the lock-free circular queue can be obtained in batches through the package sending thread. Specifically, the address of the session data can be obtained node by node, and the corresponding session data can be obtained according to the address, and the data can be packaged and sent. When "invalid" address information is encountered, it can be skipped, thereby saving packaging time and backup message space.
[0129] In some scenarios, the packing and sending thread in the primary firewall sends the packaged data to be backed up to the standby firewall through the heartbeat line between the primary firewall and the standby firewall. When the standby firewall receives the packaged data to be backed up, it parses the packaged data to be backed up through the VRRP parsing thread, and restores and backs up the data to be backed up through the backup data restoration thread.
[0130] Based on the same technical concept, see Fig.15 As shown, a data backup device 1500 provided in an embodiment of the present application can execute any step in the above data backup method. To avoid repetition, it is not described here. The device 1500 includes a first processing unit 1501 and a second processing unit 1502.
[0131] The first processing unit 1501 is used for, when determining that N processes generate target data, for each process, controlling the process to write the storage address of the target data into a first target position of the lock-free circular queue, and sending the target index position of the first target position in the lock-free circular queue to the process, so that the process saves the target index position in the target data; wherein the first target position is the position pointed to by the producer head index of the ring buffer, and N is a positive integer greater than or equal to 1;
[0132] The first processing unit 1501 is further configured to determine a target index position from the target data of the target process when it is determined that the target data of the target process needs to be deleted, and delete a storage address of the target data of the target process from the lock-free circular queue based on the target index position; the target process is any one of the N processes;
[0133] When determining that the target data needs to be sent, the second processing unit 1502 obtains the target data to be sent based on the storage address in the lock-free circular queue, and sends the target data to be sent to the backup device.
[0134] In a possible implementation, the first processing unit 1501, when controlling the process to write the storage address of the target data into the first target position of the lock-free circular queue, is specifically configured to:
[0135] Control the process to obtain a producer head index in the ring buffer and a storage object in a first target position of the lock-free ring queue pointed to by the producer head index;
[0136] When the process determines that the storage object in the position pointed to by the producer head index in the lock-free circular queue is the same as the storage object in the first target position obtained, the process is controlled to write the storage address of the target data into the first target position of the lock-free circular queue.
[0137] In a possible implementation, the first processing unit 1501 is further configured to:
[0138] Setting the index position of the preset producer's next index to the next index position of the producer's head index;
[0139] After the storage address of the target data is written into the first target position, the index position of the producer header index in the ring buffer is updated to the index position of the preset producer next index.
[0140] In a possible implementation, the first processing unit 1501 is further configured to:
[0141] When the process determines that the storage object in the position pointed to by the producer header index in the lock-free circular queue is different from the storage object in the acquired first target position, the process is controlled to re-acquire the producer header index in the circular buffer.
[0142] In a possible implementation, when the first processing unit 1501 deletes the storage address of the target data of the target process from the lock-free circular queue based on the target index position, it is specifically configured to:
[0143] Control the target process to obtain the consumer head index and the producer tail index in the ring buffer;
[0144] When it is determined that the target index position of the target process is between the consumer head index and the producer tail index, the data in the target index position in the lock-free circular queue is set to invalid, and a message indicating that the data in the target index position has been successfully deleted is returned to the target process.
[0145] In a possible implementation, the first processing unit 1501 is further configured to:
[0146] Obtaining a storage object at a second target location pointed to by the consumer head index;
[0147] Before returning a message to the target process indicating successful deletion of the data in the target index position, it is determined that the storage object in the position pointed to by the consumer head index in the lock-free circular queue is the same as the acquired storage object of the second target position.
[0148] In a possible implementation, the first processing unit 1501 is further configured to:
[0149] When it is determined that the storage object at the position pointed to by the consumer head index in the lock-free circular queue is different from the storage object at the second target position obtained, and the target index position is not between the consumer head index and the producer tail index, a message is returned to the target process to express that the data deletion in the target index position has failed.
[0150] In a possible implementation, when the second processing unit 1502 obtains the target data to be sent based on the storage address in the lock-free circular queue, it is specifically configured to:
[0151] Obtaining the consumer head index in the ring buffer and the number of nodes to be obtained;
[0152] Determine the index position of the next index of the preset consumer based on the number of nodes to be acquired and the consumer head index;
[0153] For each index position between the consumer head index and the preset consumer next index, obtain the storage address in the third target position pointed to by each index position in the lock-free circular queue, and obtain the target data to be sent based on the storage address in the third target position.
[0154] Based on the same technical concept, see Fig.16 As shown, an execution device 1600 provided in an embodiment of the present application can implement any step of the data backup method discussed above, and will not be described again here to avoid repetition. The device 1600 includes a memory 1601 and a processor 1602 .
[0155] The memory 1601 is used to store program instructions;
[0156] The processor 1602 is used to call the program instructions stored in the memory and execute any step of the above data backup method according to the obtained program.
[0157] In the embodiment of the present application, the processor 1602 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0158] Memory 1601, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1601 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic storage, disk, optical disk, etc. Memory 1601 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1601 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0159] Based on the same technical concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any step in the above-mentioned data backup method. In order to avoid repetition, it will not be repeated here.
[0160] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0161] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0162] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0164] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A data backup method, characterized in that: include: When it is determined that N processes generate target data, for each process, control the process to write the storage address of the target data into a first target position of the lock-free circular queue, and send the target index position of the first target position in the lock-free circular queue to the process, so that the process saves the target index position in the target data; wherein the first target position is the position pointed to by the producer head index of the ring buffer, and N is a positive integer greater than or equal to 1; When it is determined that the target data of the target process needs to be deleted, a target index position is determined from the target data of the target process, and a storage address of the target data of the target process is deleted from the lock-free circular queue based on the target index position; the target process is any one of the N processes; When it is determined that the target data needs to be sent, the target data to be sent is acquired based on the storage address in the lock-free circular queue, and the target data to be sent is sent to the backup device.
2. The method according to claim 1, characterized in that The controlling the process to write the storage address of the target data into the first target position of the lock-free circular queue comprises: Control the process to obtain a producer head index in the ring buffer and a storage object in a first target position of the lock-free ring queue pointed to by the producer head index; When the process determines that the storage object in the position pointed to by the producer head index in the lock-free circular queue is the same as the storage object in the first target position obtained, the process is controlled to write the storage address of the target data into the first target position of the lock-free circular queue.
3. The method according to claim 2, characterized in that The method further comprises: Setting the index position of the preset producer's next index to the next index position of the producer's head index; After the storage address of the target data is written into the first target position, the index position of the producer header index in the ring buffer is updated to the index position of the preset producer next index.
4. The method according to claim 2, characterized in that The method further comprises: When the process determines that the storage object in the position pointed to by the producer header index in the lock-free circular queue is different from the storage object in the acquired first target position, the process is controlled to re-acquire the producer header index in the circular buffer.
5. The method according to claim 1, characterized in that The deleting the storage address of the target data of the target process from the lock-free circular queue based on the target index position includes: Control the target process to obtain the consumer head index and the producer tail index in the ring buffer; When it is determined that the target index position of the target process is between the consumer head index and the producer tail index, the data in the target index position in the lock-free circular queue is set to invalid, and a message indicating that the data in the target index position has been successfully deleted is returned to the target process.
6. The method according to claim 5, characterized in that The method further comprises: Obtaining a storage object at a second target location pointed to by the consumer head index; Before returning a message to the target process indicating successful deletion of the data in the target index position, it is determined that the storage object in the position pointed to by the consumer head index in the lock-free circular queue is the same as the acquired storage object of the second target position.
7. The method according to claim 5, characterized in that The method further comprises: When it is determined that the storage object at the position pointed to by the consumer head index in the lock-free circular queue is different from the storage object at the second target position obtained, and the target index position is not between the consumer head index and the producer tail index, a message is returned to the target process to express that the data deletion in the target index position has failed.
8. The method according to any one of claims 1 to 7, characterized in that: The acquiring the target data to be sent based on the storage address in the lock-free circular queue includes: Obtaining the consumer head index in the ring buffer and the number of nodes to be obtained; Determine the index position of the next index of the preset consumer based on the number of nodes to be acquired and the consumer head index; For each index position between the consumer head index and the preset consumer next index, obtain the storage address in the third target position pointed to by each index position in the lock-free circular queue, and obtain the target data to be sent based on the storage address in the third target position.
9. A data backup device, characterized in that: include: A first processing unit is used for, when determining that N processes generate target data, for each process, controlling the process to write the storage address of the target data into a first target position of the lock-free circular queue, and sending the target index position of the first target position in the lock-free circular queue to the process, so that the process saves the target index position in the target data; wherein the first target position is a position pointed to by a producer head index of the ring buffer, and N is a positive integer greater than or equal to 1; The first processing unit is further configured to determine a target index position from the target data of the target process when it is determined that the target data of the target process needs to be deleted, and delete a storage address of the target data of the target process from the lock-free circular queue based on the target index position; the target process is any one of the N processes; The second processing unit, when determining that the target data needs to be sent, obtains the target data to be sent based on the storage address in the lock-free circular queue, and sends the target data to be sent to the backup device.
10. An execution device, characterized in that: include: A memory for storing program instructions; A processor is used to obtain program instructions stored in the memory and execute the method according to any one of claims 1 to 8 according to the program instructions.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer is enabled to execute the method according to any one of claims 1 to 8.