Data synchronization method, device, equipment and medium based on FTTR / PON / WIFI
By using the minimum perfect hashing algorithm and UDP packet notification mechanism in the FTTR/PON/WIFI network, the real-time and complexity of data synchronization are solved, and efficient inter-process communication and cross-device data synchronization are achieved.
Patent Information
- Application Number
- CN202411178923.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-27
AI Technical Summary
The prior art has poor real-time and complex methods in terms of data synchronization, especially in inter-process communication within network devices and cross-device process communication, where the message queue mechanism leads to delays and complexity.
Using the data synchronization method based on FTTR/PON/WIFI, the target key is queryed by using the minimum perfect hash algorithm in the shared memory area, and using UDP packets to notify the subscription process of data updates, realizing direct communication between processes, replacing the traditional message queue mechanism .
Improves the real-time and efficiency of data synchronization, simplifies inter-process communication, reduces latency, and supports multi-process subscription mode to realize a one-to-many communication mode.
Smart Images

Figure CN118694465B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology and provides a data synchronization method, device, equipment and medium based on FTTR / PON / WIFI. Background Art
[0002] In traditional local area networks, the existing methods for data synchronization are relatively complex, and the inter-process communication within network devices and the inter-process communication across devices usually adopt the message queue mechanism. The message queue is usually first-in-first-out. After the message is sent, it needs to wait for the receiver to actively pull or poll, which has a certain delay, resulting in poor real-time data synchronization. Therefore, the existing technology has the problems of poor real-time data synchronization and relatively complex methods. Summary of the invention
[0003] The present application provides a data synchronization method, device, equipment and medium based on FTTR / PON / WIFI, which are used to solve the problems of poor real-time performance and relatively complex methods of data synchronization in the prior art.
[0004] In a first aspect, a data synchronization method based on FTTR / PON / WIFI equipment is provided, which is applied to any FTTR / PON / WIFI equipment; the shared memory of any equipment includes a public shared memory area and a shared memory area of the equipment; the public shared memory area pre-stores a plurality of fixed keys; the method comprises:
[0005] Starting a daemon thread to receive a data write request initiated by the first process; the data write request is used to request to write target data;
[0006] Using a minimum perfect hash algorithm to operate on a target key corresponding to the target data to obtain a first offset address;
[0007] If the key pointed to by the first memory address in the public shared memory area is the same as the target key, the second offset address corresponding to the target key is queried, and the target data is written into the shared memory area of the local device according to the second memory address; the first memory address is the sum of the first address of the public shared memory area and the first offset address; the second memory address is the sum of the first address of the shared memory area of the local device and the second offset address;
[0008] Sending a UDP data packet to a second process that has subscribed to the target key; the UDP data packet is used to indicate that the data pointed to by the offset address corresponding to the target key is updated;
[0009] In response to the query request of the second process, the target data is read from the shared memory area of the local device.
[0010] Optionally, before sending the UDP data packet to the second process subscribed to the target key, the method further includes:
[0011] Receive a registration request from a second process; the registration request includes port information of a listening port and at least one key to be subscribed; the at least one key includes a target key; the listening port is an idle UDP port requested by the second process to the Linux system;
[0012] registering the second process according to the at least one key and the port information of the listening port;
[0013] The sending of a UDP data packet to a second process subscribed to the target key comprises:
[0014] A UDP data packet is constructed with the listening port as the destination port and the IP of the device where the second process is located as the destination IP, and the UDP data packet is sent to the second process.
[0015] Optionally, writing the target data into the shared memory area of the local device according to the second memory address includes:
[0016] Acquire a third offset address according to the second memory address, and add the first address of the shared memory area of the local device to the third offset address to obtain a third memory address;
[0017] According to the third memory address and the data length corresponding to the target key, querying the original data from the shared memory area of the local device;
[0018] Determining whether the target data is different from the original data;
[0019] If so, the memory pointed to by the third memory address is released, new memory is applied for according to the length of the target data, and the target data is written into the new memory.
[0020] Optionally, the shared memory area of the device stores data according to a preset data structure; the data structure uses a version number, a timestamp, and a data length as a data header, and the data itself as a data body; the initial value of the version number is 0;
[0021] Writing the target data into the new memory comprises:
[0022] Add 1 to the version number, update the timestamp and the data length, write the target data into the new memory, and then add 1 to the version number again;
[0023] The step of reading the target data from the shared memory area of the local device in response to the query request of the second process includes:
[0024] receiving and parsing a query request of the second process to obtain the target key; the query request includes the target key;
[0025] According to the target key, obtain the version number;
[0026] If the version number is an odd number, continue to wait; if the version number is an even number, read the target data from the shared memory area of the local device.
[0027] Optionally, the any device is the FTTR device, and the shared memory of the any device includes a copy data area; after writing the target data into the shared memory area of the device according to the second memory address, the method further includes:
[0028] The target key, the length of the target data, and the updated offset address are stored in a synchronization linked list;
[0029] Start the inter-device data synchronization thread, and if the length of the synchronization linked list is greater than a preset threshold, obtain the partition lock of any of the devices, and copy all the data in the shared memory area of the device to the copied data area;
[0030] The partition lock is released, and the data in the copied data area is sent to other devices of the FTTR.
[0031] Optionally, sending the data in the copied data area to other devices of the FTTR includes:
[0032] Encapsulating the data in the copy data area according to a preset packet format to obtain encapsulated data; the packet format includes a destination MAC address, a source MAC address, a protocol number, a packet type, a partition number, and a payload block;
[0033] The driver layer is called to send the encapsulated data to other devices of the FTTR.
[0034] Optionally, starting a synchronization data thread between devices includes:
[0035] If the frequency of writing data within the preset time period is greater than or equal to the preset frequency, the inter-device data synchronization thread is started regularly;
[0036] If the frequency of writing data within the preset time period is less than the preset frequency, the inter-device data synchronization thread is immediately started.
[0037] In a second aspect, a data synchronization device based on FTTR / PON / WIFI equipment is provided, wherein the device is arranged in any device of FTTR / PON / WIFI; the shared memory of any device comprises a public shared memory area and a shared memory area of the device; the public shared memory area pre-stores a plurality of fixed keys; the device comprises:
[0038] A receiving module, used to start a daemon thread and receive a data write request initiated by a first process; the data write request is used to request to write target data;
[0039] A calculation module, used for calculating the target key corresponding to the target data by using a minimum perfect hash algorithm to obtain a first offset address;
[0040] A writing module, used for, if the key pointed to by the first memory address in the public shared memory area is the same as the target key, querying the second offset address corresponding to the target key, and writing the target data into the shared memory area of the local device according to the second memory address; the first memory address is the sum of the first address of the public shared memory area and the first offset address; the second memory address is the sum of the first address of the shared memory area of the local device and the second offset address;
[0041] A sending module, used to send a UDP data packet to a second process subscribed to the target key; the UDP data packet is used to indicate that the data pointed to by the offset address corresponding to the target key is updated;
[0042] The reading module is used to read the target data from the shared memory area of the local device in response to the query request of the second process.
[0043] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the FTTR / PON / WIFI-based data synchronization method described in the first aspect.
[0044] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and a processor executes the computer program to implement the FTTR / PON / WIFI-based data synchronization method described in the first aspect.
[0045] Compared with the prior art, the present invention has the following beneficial effects:
[0046] The present application provides a data synchronization method based on FTTR / PON / WIFI, which is applied to any device of FTTR / PON / WIFI; the shared memory of any device includes a public shared memory area and a shared memory area of the device; the public shared memory area stores multiple fixed keys in advance; the shared memory area of the device is used to store the data of any device. Before writing the target data into the shared memory area of the device, the method uses a minimum perfect hash algorithm to query whether there is a target key corresponding to the target data in the public shared memory area. The minimum perfect hash algorithm can map the unique key to different positions in the hash table without collision, and the size of the hash table is exactly equal to the number of key sets, without redundant space, which can reduce the search time of the hash table, thereby quickly finding the target key, thereby improving the real-time performance of subsequent data synchronization. Further, after writing the target data, a UDP data packet is sent to notify the second process that subscribes to the target key, so that the second process is informed in time that the offset address corresponding to the target key is updated, and the newly written target data is read from the shared memory area of the device. This message subscription mechanism can completely replace the communication mechanism of the inter-process message queue, making the inter-process communication within the same device or across devices simple, efficient, and with less delay. The message subscription mechanism supports multiple processes to subscribe to the same key at the same time. When multiple second processes subscribe to the target key, UDP data packets can be received by these multiple second processes, realizing a one-to-many communication mode and improving the real-time performance of data synchronization. The data writing process is completed by the daemon thread. One thread completes all operations of writing data. There is no competition for writing, so there is no need to use lock protection, skipping the process of applying for and releasing locks, and there is no lock blocking, further improving the real-time performance of data synchronization. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0048] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;
[0049] Figure 2 A schematic diagram of the partitioning of four FTTR devices provided in the embodiment of the present application;
[0050] Figure 3 A flow chart of a data synchronization method based on FTTR / PON / WIFI provided in an embodiment of the present application;
[0051] Figure 4A schematic diagram of a process for writing a value corresponding to a key provided in an embodiment of the present application;
[0052] Figure 5 A schematic diagram of a data structure provided in an embodiment of the present application;
[0053] Figure 6 A schematic diagram of a UDP data packet provided in an embodiment of the present application;
[0054] Figure 7 A schematic diagram of a process for reading the value corresponding to a key provided in an embodiment of the present application;
[0055] Figure 8 A schematic diagram of a read process of a lock-free data structure provided in an embodiment of the present application;
[0056] Fig. 9 Schematic diagram of multiple DataSyn systems provided in embodiments of the present application;
[0057] Fig.10 A schematic diagram of the overall architecture of the DataSyn system provided in the embodiment of the present application;
[0058] Fig.11 A schematic diagram of the structure of a data synchronization device based on FTTR / PON / WIFI provided in an embodiment of the present application. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0060] In order to solve the problems of poor real-time performance and relatively complex methods of data synchronization in the prior art, the embodiment of the present application provides a data synchronization method based on FTTR / PON / WIFI, which can be applied to any device of FTTR / PON / WIFI. Any device can be a fiber to the room (Fiber To The Room, FTTR) device, the FTTR device includes a main gateway and a sub-gateway, any device can be a passive optical network (Passive Optical Network, PON) device, the PON device includes a (Gigabit Passive Optical Network, GPON) gateway and an (Ethernet Passive Optical Network, EPON) gateway, and any device can also be a WIFI router.
[0061] Please refer to Figure 1 , is a schematic diagram of an application scenario provided by an embodiment of the present application, or can be understood as a FTTR network topology diagram. The FTTR network includes a main gateway and several sub-gateways. The sub-gateways are connected to the main gateway through optical fibers. The sub-gateways and the main gateway can communicate directly, and the communication between the sub-gateways must go through the main gateway. Any device can be the main gateway or any sub-gateway.
[0062] The embodiment of the present application provides a data synchronization DataSyn (data synchronization) system. During the system initialization process of any device, a kernel module DataSyn.ko, also known as a DataSyn module, is loaded into the kernel. For FTTR networks, the DataSyn system refers to a system composed of DataSyn modules of all FTTR devices in the FTTR network. For PON and WIFI, the DataSyn system refers to a system composed of a single DataSyn module in the device itself.
[0063] During the device initialization process, the DataSyn module is started as soon as possible, and a piece of memory with a continuous physical address space is allocated through the memory allocation function kmalloc. This memory is called shared memory. Because the DataSyn module is loaded early enough, it can be guaranteed that the memory allocated using kmalloc is physically continuous. The continuous physical memory area can be more efficiently used by the CPU cache system, improving the cache hit rate, reducing the latency of memory access, and improving performance.
[0064] The DataSyn module initializes the shared memory and divides it into several partitions. Each partition is used to store data of different functional modules. The size of each partition can be the same or different. For each partition, the key-length-value structure is used to organize the data. The key is a string used to identify the data ID, the length indicates the length of the data, and the value is the offset address of the stored data, which can accommodate text or binary data.
[0065] If any device is a PON device and a WIFI router, the partitions only include the public shared memory area and the local device shared memory area. The public shared memory area stores multiple fixed keys in advance, and these keys cannot be changed once fixed. The local device shared memory area is used to store the data of any device and the offset address of these data.
[0066] If any device is an FTTR device, several partitions include a public shared memory area, a shared memory area for this device, and shared memory areas for multiple other devices. The public shared memory area also stores the correspondence between the MAC addresses of multiple FTTR devices and the partition numbers. The partition number can be queried in the public shared memory area based on the MAC address. Each other device shared memory area is used to store the data of each other device and the offset address of these data. Other devices are FTTR devices other than the device in the FTTR network. For example, if the FTTR network has N FTTR devices, it is divided into N+2 partitions, numbered from area 0 to area N+1. Among them, area 0 is used as a public shared memory area, area N+1 is used as a replicated data area, area 1 is used as a partition dedicated to the main gateway, and areas 2 to N are used as partitions dedicated to sub-gateways.
[0067] Please refer to Figure 2 , which is a partition diagram of four FTTR devices provided in an embodiment of the present application. The four FTTR devices include a main gateway and three sub-gateways, zone 0 is a public shared memory zone, zone 1 is a shared memory zone for device No. 1, zone 1 is dedicated to the main gateway, and zones 2-4 are dedicated to the three sub-gateways, among which zone 2 is a shared memory zone for device No. 2, zone 3 is a shared memory zone for device No. 3, and zone 4 is a shared memory zone for device No. 4. Zone N+1 is a replicated data zone, which is dedicated to the synchronization data thread between devices.
[0068] Because each sub-gateway is connected to the main gateway by optical fiber, the partition of the main gateway is located in the front, and the remaining partitions are assigned to the sub-gateways by the main gateway. All partitions can be the same size or different. The partitions of each FTTR device are consistent, and the data stored in each partition is also consistent after full synchronization, so that each FTTR device has all the data shared by the entire FTTR network device, and the query data only needs to be queried in the shared memory of this device.
[0069] In the partition data, there are a large number of pointers, because the process in the user space will map the shared memory to different memory addresses. In order to adapt to different address spaces, the pointer value stored in the shared memory is the difference between the actual address of the kernel space and the first address of the shared memory. In the user space, no matter how much the mapped address space is, the address offset will not change. You only need to add this pointer value to the first address of the shared memory partition to get the actual address in the user space. For example, the starting address of a partition in the kernel space is 0xC0100000, and there is a pointer that originally points to 0xC0108000. When storing the pointer value, take the difference between the two (0xC0108000-0xC0100000=0x8000), add the pointer value to the first address of the shared memory partition in the user space 0xB0200000, and get the address that the pointer actually points to is 0xB0208000.
[0070] As described above, the application scenarios of the embodiments of the present application are described below. Figure 3 The data synchronization method based on FTTR / PON / WIFI is introduced as shown.
[0071] S301, start a daemon thread and receive a data write request initiated by a first process.
[0072] In the specific implementation process, after the DataSyn module of any device completes the partitioning, it wakes up the daemon thread and waits for the request from the user space of any device to write data. There are multiple processes in the user space. When the first process wants to write the target data, it initiates a data write request. The data write request is used to request to write the target data.
[0073] In a possible embodiment, the DataSyn module provides a dynamic library. When receiving a data write request initiated by the first process, the dynamic library is loaded. The dynamic library is used to initialize a shared memory of a preset size and map it to the kernel space. The preset size is, for example, 4096 bytes (assuming that it is the size of a page of memory in this system). The DataSyn module applies for a system call number, writes the target data into the shared memory of the preset size in a key-length-value structure, starts a daemon thread through the system call, and the daemon thread receives the target data from the shared memory.
[0074] S302: Use a minimum perfect hash algorithm to operate on a target key corresponding to the target data to obtain a first offset address.
[0075] In the specific implementation process, the public shared memory area stores multiple fixed keys. The key is defined as any value in the range of 1-256 bytes. All keys are planned in advance, and different keys are set in advance according to different data types. No keys are added or deleted in the middle. All read and write operations are indexed based on key-value pairs. Therefore, after obtaining the target data, the target key corresponding to the target data can be queried according to the data type of the target data. The target key is subjected to the minimum perfect hash operation, and the calculated hash value is the first offset address. The first offset address is the offset value relative to the first address of the public shared memory area.
[0076] S303: If the key pointed to by the first memory address in the public shared memory area is the same as the target key, a second offset address corresponding to the target key is queried, and the target data is written into the shared memory area of the local device according to the second memory address.
[0077] The first memory address is the sum of the first address of the public shared memory area and the first offset address; the second memory address is the sum of the first address of the shared memory area of the local device and the second offset address.
[0078] In the specific implementation process, the first address of the shared memory area of the device is added to the second offset address to obtain the first memory address, and the corresponding key is read from the public shared memory area according to the first memory address to determine whether the key is the same as the target key. If they are different, it means that the target key does not exist in the public shared memory area, and then exit. If they are the same, the second offset address corresponding to the target key is queried, and the first address of the shared memory area of the device is added to the second offset address to obtain the second memory address.
[0079] In a possible embodiment, the step of writing the target data into the shared memory area of the device according to the second memory address includes:
[0080] Obtain a third offset address according to the second memory address, add the first address of the shared memory area of the device to the third offset address, and obtain the third memory address; query the original data from the shared memory area of the device according to the third memory address and the data length corresponding to the target key; determine whether the target data is different from the original data; if so, release the memory pointed to by the third memory address, apply for new memory according to the length of the target data, and write the target data into the new memory.
[0081] In the embodiment of the present application, the third offset address is obtained in the memory pointed to by the second memory address, and the first address of the shared memory area of the device is added to the third offset address to obtain the third memory address, which is the actual address where value is stored. If the target data and the original data originally stored at the third memory address are exactly the same, there is no need to write the target data again. If the target data and the original data originally stored at the third memory address are different, the target data is written again. By comparing the data information, repeated writing can be avoided, thereby avoiding unnecessary disk I / O operations and data transmission, saving system resources and time, and reducing unnecessary write operations can reduce the system load, enabling the system to process other tasks faster, and improving the system performance and response speed.
[0082] Please refer to Figure 4 , which is a flow chart of writing the value corresponding to a key provided in an embodiment of the present application.
[0083] First, calculate the HASH value (i.e., the first offset address) of the key through the minimum perfect hash operation, and compare the key string at the HASH value position in zone 0 (i.e., the first memory address) to see if it matches. If it does not match, exit abnormally. If it matches, obtain the offset pointer value (i.e., the second offset address) of the value corresponding to the key. Search in non-zone 0, query the corresponding N zone according to the MAC address of the device, and the first address of zone N plus the offset pointer value is the new memory address (i.e., the second memory address). The value found in the new memory address (i.e., the third offset address) plus the first address of zone N is the actual address where the value is stored (i.e., the third memory address), release the block of memory (i.e., the memory pointed to by the third memory address), apply for new memory and write data, and update the value data.
[0084] For example, the value of the devlist.info key calculated by the minimum perfect hash operation is 0x8090 (the offset value relative to the first address of zone 0). After the offset address of zone 0 is verified, it is the devlist.info key that matches, and the pointer to the value is stored at the offset address of 0x2c9060. If it is a PON device and a WIFI router, there is only one local device shared memory area. The first address of the local device shared memory area plus 0x2c9060 is used to obtain the second memory address, and the third offset address is obtained in the memory pointed to by the second memory address. The third offset address is added to the first address of the local device shared memory area to obtain the third memory address, which is the actual address where the value is stored.
[0085] If it is an FTTR device, there is a shared memory area for this device and multiple shared memory areas for other devices. The corresponding partition is queried according to the MAC address of the device. For device No. 1, assuming that the first address of zone 1 is 0x10000000, the second memory address is: 0x10000000+0x2c9060=0x102c9060. The third offset address is obtained in the memory pointed to by 0x102c9060. The third offset address is added to the first address of zone 1 0x10000000 to obtain the third memory address. For device No. 2, assuming that the first address of zone 2 is 0x20000000, then 0x20000000+0x29060=0x202c9060. The third offset address is obtained in the memory pointed to by 0x202c9060. The third offset address is added to the first address of zone 2 0x20000000 to obtain the third memory address, which is the actual address where value is stored.
[0086] In the embodiments of the present application, offset addresses are used. The storage addresses of the same data in the actual memory space of different devices may be different, but the offset addresses are the same. In other words, the data stored in the device shared memory area may change, but all keys stored in the public shared memory area and all offset addresses stored in the device shared memory area remain unchanged.
[0087] In a possible embodiment, in order to ensure high efficiency and real-time, the DataSyn system synchronizes and communicates small blocks of data. Therefore, the DataSyn system does not allow allocation of large blocks of memory in the shared memory, and the maximum does not exceed 1024 bytes.
[0088] In a possible embodiment, the memory of each device shared partition can be divided into several groups in advance, and each group is divided into fixed-size memory blocks, for example: 256 of 1024 bytes in the first group, 1024 of 256 bytes in the second group, and 8192 of 64 bytes in the third group. Once divided, the number and size of the memory blocks will no longer change. When applying for memory, find the most suitable size of memory. If the current group has been exhausted, find a larger group of memory. The offset pointer of each group of free memory (all relative to the offset address of the N zone head) is stored in a free linked list. Memory is applied from the head of the linked list, and released memory is returned to the end of the linked list.
[0089] In the embodiment of the present application, for special consideration of performance optimization, the released memory is returned to the end of the linked list instead of the head of the linked list, because a piece of memory that has just been released is being read by a process in the user space with a very small probability. As long as the memory is not immediately written with new data, the data consistency of the memory is still guaranteed, and the validity of the read can be guaranteed with a high probability. If it is placed at the head of the linked list, the next time the same size of memory is requested, the data will be written immediately, destroying the consistency of the data. Placing it at the end of the linked list greatly reduces this probability and improves performance.
[0090] In a possible embodiment, the shared memory area of the device stores data according to a preset data structure; the data structure uses a version number, a timestamp, and a data length as a data header, and the data itself as a data body; the initial value of the version number is 0.
[0091] Please refer to Figure 5 , which is a schematic diagram of a data structure provided in an embodiment of the present application. The data structure includes a data header and a data body, the data header includes a 4-byte version number, a 4-byte timestamp, and a 4-byte data length, and the data body includes several bytes of data. In an embodiment of the present application, the DataSyn module provides a lock-free data structure to the process in the user space, so that the second process can read consistent data without locking when reading data.
[0092] S304: Send a UDP data packet to the second process that has subscribed to the target key.
[0093] In a possible embodiment, the DataSyn module can receive a registration request from the second process; the registration request includes the port information of the listening port and at least one key to be subscribed; the at least one key includes a target key; the listening port is an idle UDP port requested by the second process to the Linux system; the second process is registered according to the port information of the at least one key and the listening port. Further, after writing the target data, a UDP data packet is constructed with the listening port as the destination port and the IP of the device where the second process is located as the destination IP, and the UDP data packet is sent to the second process. The UDP data packet is used to indicate that the data pointed to by the offset address corresponding to the target key is updated.
[0094] In the specific implementation process, if any device is a PON device or a WIFI router, the second process can only be the user space process of any device. If any device is an FTTR device, the second process can be the user space process of any device, or the user space process of other devices in the FTTR. The second process can apply for a UDP listening port from the Linux system in advance (assuming that the idle UDP port applied for is 30819). The specific method is to create a UDP socket, bind the loopback address or 0 address (0.0.0.0) and 0 port. At this time, the Linux system will assign an idle and unused listening port to the second process. The second process initiates registration with the DataSyn module based on the port information of the listening port and the key of interest.
[0095] Furthermore, when the DataSyn module detects that the data pointed to by the offset address corresponding to the target key changes (when the daemon thread writes the target data), the daemon thread uses the IP of any device as the source IP, any port of any device as the source port, the listening port (for example, the idle UDP port 30819 applied for) as the destination port, and the IP of the device where the second process is located as the destination IP, constructs a UDP data packet, and sends the UDP data packet to the second process. If the second process is a user space process of any device, the destination IP is the local loopback address; if the second process is a user space process of other devices, the destination IP is the IP of other devices.
[0096] Please refer to Figure 6 , which is a schematic diagram of a UDP data packet provided in an embodiment of the present application. The UDP packet payload is data encapsulated in the DataSyn format. For process communication within a device, the destination port is filled in with the destination port of the process, the source port is an arbitrary port (for example, 0), the destination IP is filled in with 127.0.0.1, i.e., the local loopback address, and the source IP is filled in with the IP of this device. For process communication between devices, the destination port is filled in with the destination port of the process, the source port is an arbitrary port (for example, 0), the destination IP is filled in with the IP of the destination device (i.e., other devices), and the source IP is filled in with the IP of this device.
[0097] Figure 6 The packet content of each layer involved is filled in by the daemon thread. Among them, the checksum of the UDP packet does not need to be calculated and can be directly filled with 0, because the probability of bit errors in communication within and between devices is almost zero, which can further improve efficiency.
[0098] In an embodiment of the present application, UDP data packets are constructed with different destination IPs according to the different second processes to ensure that different processes can receive UDP data packets.
[0099] The following describes how the daemon thread sends UDP packets.
[0100] The daemon thread can apply for a sk_buff structure in the Linux kernel to store the data packets to be sent, then construct the IP header and UDP header, fill in the relevant information of the IP header and UDP header in the sk_buff structure, including the source port, destination port, source IP, and destination IP, copy the data to be sent to the data area in the sk_buff structure, and call the ip_local_out function provided by the Linux kernel to send the UDP data packet. The data to be sent includes the target key and the updated offset address, and the updated offset address is the offset address where the target data is actually stored.
[0101] In an embodiment of the present application, a UDP data packet is sent through the interface function ip_local_out provided by the Linux kernel. Under normal circumstances, it takes no more than a few microseconds. Even if the second process does not read the socket data in time, it will not cause the daemon thread to be blocked. After the UDP packet is sent out, it is further processed by the Linux kernel protocol stack. The Linux kernel protocol stack has multiple mechanisms to ensure the efficiency of UDP packet sending and receiving data.
[0102] Considering that some Linux kernel versions do not support sending UDP packets through the ip_local_out function, the kernel code can be modified. The daemon thread can first create a UDP packet to be sent, fill in the relevant information of the IP header and the UDP header, add the constructed packet to the sending queue of the TCP / IP protocol stack, and then return immediately. The network protocol stack of the Linux kernel continues to process the packet in the original way and finally sends the packet to the second process. In this way, even if some Linux kernel versions do not support directly calling the ip_local_out function to send UDP packets, it is possible to send UDP packets without blocking the packet sending thread.
[0103] Among them, the format of UDP data packet is as follows:
[0104] Table 1
[0105]
[0106] If the length of the payload is equal to 256 bytes, the UDP data packet does not include the Value data. If the length of the Payload is greater than 256 bytes, the UDP data packet includes the Value data, and the actual length of the Value is the Payload length minus 256. In other words, the daemon thread can encapsulate the target key and the updated offset address together in a UDP data packet and send it to the second process. The daemon thread can also only encapsulate the target key in a UDP data packet and send it to the second process, and the second process will query the updated offset address based on the target key. In the DataSyn module, the maximum length of the data is limited to 1024 bytes, so the Payload of a UDP packet can meet the requirements.
[0107] After the second process receives the UDP data packet from the socket, it first checks whether the magic number field matches. If the match is successful, it extracts the key (target key) and Value (updated offset address) in the UDP data packet.
[0108] S305. In response to the query request of the second process, read the target data from the shared memory area of the device.
[0109] Please refer to Figure 7 , which is a flow chart of reading the value corresponding to a key provided in an embodiment of the present application.
[0110] First, the HASH value (i.e., the first offset address) of the key is calculated using the minimum perfect hash algorithm. The key string is compared at the HASH value position in zone 0 (i.e., the first memory address) to see if it matches. If it does not match, the program exits abnormally. If it matches, the offset pointer value (i.e., the second offset address) of the value corresponding to the key is obtained. In non-zone 0, the corresponding N zone is queried based on the MAC address of the device. The first address of zone N plus the offset pointer value is the new memory address (i.e., the second memory address). The value found in the new memory address (i.e., the third offset address) plus the first address of zone N is the actual memory address where the value is stored (i.e., the third memory address). The value is obtained at the actual memory address where the value is stored.
[0111] In a possible embodiment, the step of writing the target data into the new memory includes: adding 1 to the version number, updating the timestamp and the data length, and after writing the target data into the new memory, adding 1 to the version number. In response to the query request of the second process, the step of reading the target data from the shared memory area of the current device includes: receiving and parsing the query request of the second process to obtain the target key; the query request includes the target key; according to the target key, obtaining the version number; if the version number is an odd number, continuing to wait; if the version number is an even number, reading the target data from the shared memory area of the current device.
[0112] In the specific implementation process, first increase the version number of this memory area by 1, update the data length, timestamp and other information, and increase the version number by 1 again after writing the target data. Because the initial value of the version number is 0, if the version number is an odd number, it means that the DataSyn module is writing data (not yet completed), and if the version number is an even number, it means that the data has been written. After receiving the query request from the second process, the DataSyn module parses to obtain the target key, queries the updated offset address corresponding to the target key, adds the first address of the shared memory area of this device to the updated offset address, obtains the third memory address, and reads the version number from the shared memory area of this device according to the third memory address.
[0113] If the version number is an odd number, you can use sleep(0) to temporarily give up the CPU until the version number is confirmed to be an even number, and immediately copy all the data (including the version number) from the shared memory area of this device. If the first process is the user space process of any device, all the data will be copied to the user space of any device, and the version number of the user space copied data will be compared with the version number of the shared memory data area of this device. If they are inconsistent, it means that the block of data has been rewritten by the DataSyn module, and the version number is obtained again according to the target key until the version number is consistent. The version number only needs to consider whether it is the same or different, and only needs a long period of time (for example, within 60 seconds) without duplication (the 4-byte version number is more than enough), so there is no need to consider the overflow problem of the increasing version number. Because the probability of conflict due to read-write competition is extremely low, in most cases, the performance is far better than the lock protection method.
[0114] Please refer to Figure 8 , which is a schematic diagram of the reading process of the lock-free data structure provided in an embodiment of the present application.
[0115] First, start querying the value of key, obtain the value pointed to by key (i.e., the second offset address), obtain the version number of value according to the second offset address, and determine whether the version number is an odd number. If the version number is an odd number, use sleep(0) to temporarily give up the CPU scheduling right and continue to obtain the value pointed to by key. If the version number is not an odd number (it is an even number), copy the data and compare the version number, timestamp and other information of the copied data with the original data to see if they are consistent. If they are consistent, the copy is successful and the process ends. If they are inconsistent, continue to obtain the value pointed to by key.
[0116] In an embodiment of the present application, the process in the user space uses neither locks nor system calls when reading data. Using system calls will cause switching between user space and kernel space, increasing the latency of context switching. Therefore, data is read directly from the shared memory, with particularly high efficiency and performance, especially when the number of reads is much greater than the number of writes, the performance is even more outstanding.
[0117] In a possible embodiment, if any device is an FTTR device, after any device writes the target data into the shared memory area of the device according to the second memory address, it can also store the target key, the length of the target data, and the updated offset address into the synchronization linked list, and start the data synchronization thread between devices; if the length of the synchronization linked list is greater than a preset threshold, the partition lock of any device is obtained, and all data in the shared memory area of the device is copied to the copied data area; the partition lock is released, and the data in the copied data area is sent to other devices of the FTTR.
[0118] In the specific implementation process, after writing the target data, the daemon thread puts the target key, the length of the target data, and the updated offset address in the synchronization linked list. The updated offset address is the offset address where the target data is actually stored. The synchronization linked list is a linked list queue. The daemon thread appends data to the end of the linked list. The inter-device synchronization thread takes the data from the head of the linked list and synchronizes it to other devices until the linked list is empty. If any device is the main gateway, the other devices are sub-gateways, and the main gateway synchronizes the data to each sub-gateway. If any device is a sub-gateway, the other devices are the main gateway. The sub-gateway first synchronizes the data to the main gateway, and the main gateway then synchronizes the data to other sub-gateways.
[0119] If the length of the synchronization list is less than or equal to the preset threshold, the list is copied to copy all the data in the synchronization list to the copy data area, and then the data in the copy data area is sent to other devices. The data synchronization thread can enable jumbo frames to synchronize data between devices more efficiently. Jumbo frames can break through the 1500-byte limit of Ethernet frames and accommodate more synchronization data in one frame, making synchronization more efficient.
[0120] If the length of the synchronization list is greater than the preset threshold, the entire partition is replicated to synchronize all data in the shared memory area of the device to other devices. Even if jumbo frames are used to send data, one jumbo frame cannot contain all the data of the entire partition. The data can be cut into multiple jumbo frames and then sent out.
[0121] In the embodiment of the present application, the partition lock of the device is obtained to prevent the daemon thread from writing new data, copy all the data in the shared memory area of the entire device to the copy data area, and then release the partition lock. The partition lock can be occupied as short as possible so that the daemon process can write data in real time. The benefit of keeping each partition the same size is reflected here. Each partition can be completely copied and then the copied data is sent out. When a new FTTR device joins the FTTR network, the new FTTR device has no historical data of other devices at all. Using overall partition replication can synchronize to the latest status at the fastest speed.
[0122] In a possible embodiment, the period for starting the inter-device data synchronization thread is divided into two cases. If the frequency of writing data within a preset time period is greater than or equal to the preset frequency, the inter-device data synchronization thread is started periodically; if the frequency of writing data within the preset time period is less than the preset frequency, the inter-device data synchronization thread is started immediately.
[0123] In the embodiment of the present application, the preset time period is, for example, 500ms, and the ratio between the number of data written in the preset time period and the preset time period is used as the frequency of writing data. If the frequency of writing data in the preset time period is greater than or equal to the preset frequency, it means that the writing is too frequent. Regularly waking up the data synchronization thread between devices can avoid frequent data synchronization and improve system efficiency. If the frequency of writing data is less than the preset frequency, it means that the writing frequency is low. Immediately waking up the data synchronization thread between devices can reduce the risk of data loss and ensure that the data of each device in the system remains consistent.
[0124] In a possible embodiment, the step of sending the data in the duplicate data area to other devices of the FTTR includes:
[0125] The data in the copied data area is encapsulated according to a preset packet format to obtain encapsulated data; the packet format includes a destination MAC address, a source MAC address, a protocol number, a packet type, a partition number, and a payload block; and the driver layer is called to send the encapsulated data to other devices of the FTTR.
[0126] In the embodiment of the present application, the data synchronization thread customizes the packet format and directly calls the driver layer to send and receive packets. The data is directly established on the MAC layer, bypassing the Linux TCP / IP protocol stack, so that the efficiency of sending and receiving packets is further improved.
[0127] For example, the default packet format is shown in Table 2:
[0128] Table 2
[0129]
[0130] The payload block consists of a DataSyn header and data in an ALV structure (multiple nodes can be taken out from the linked list at one time), where A stands for Address, indicating the offset value of the data relative to the partition's first address, L stands for data length (Length), and V stands for data (Value), all of which are 4-byte aligned.
[0131] The package types are shown in the following table:
[0132] Table 3
[0133]
[0134] When the sub-gateway sends a packet with a DataSyn protocol number to the main gateway, it uses the interface where the upstream optical fiber is located. After the main gateway receives the packet with a DataSyn protocol number sent by each sub-gateway, it records and updates the interface and maintains a simple MAC address interface table. The MAC address interface table includes multiple MAC addresses and the interface corresponding to each MAC address. Because there will not be many sub-gateways, the entries in this table are single digits, generally not more than 10. In this way, when both parties send packets, they do not need to look up the MAC address table, which improves the efficiency of packet sending.
[0135] In a possible embodiment, the inter-device data synchronization thread can also receive data packets from other devices. If the MAC layer protocol number of the data packet is determined to be the target protocol number (for example, 0x4358), the data packet is parsed to obtain the synchronization data and the target MAC address of the other device; the target partition number corresponding to the target MAC address is queried from the public shared memory area, the target device shared memory area corresponding to the target partition number is determined from multiple other device shared memory areas, and the synchronization data is stored in the target device shared memory area.
[0136] In the specific implementation process, in order to use the DataSyn system, multiple sub-gateways send device information (including MAC address and IP address) to the main gateway for registration at startup. The DataSyn module of the main gateway allocates a device shared memory area to each sub-gateway and stores the one-to-one correspondence between the MAC address and the partition number in the public shared memory area.
[0137] In the embodiment of the present application, the daemon process writes to the shared memory area of the device, and the data synchronization thread between devices writes to the shared memory area of other devices. The written areas are different, and there will be no write contention conflict, and no locking is required. The multi-core and multi-threaded functions of the CPU can be fully utilized to improve performance.
[0138] In a possible embodiment, the data synchronization thread between devices can confirm whether the other party has received the packet based on the sequence number of the DataSyn packet header. If not, retransmit. Each time a packet is sent, the sequence number is increased by 1. Only after confirming that the other party has received the packet, the next packet is sent. If the other party's confirmation packet is not received within 1 millisecond, retransmission is performed. Since FTTR devices are connected via high-speed optical fiber, packet loss is not easy, and the possibility of retransmission is very small, so communication efficiency is guaranteed.
[0139] When two processes need to communicate frequently and with large amounts of data, it is not appropriate to use a single DataSyn system. Writing data at a very high frequency will take up a large amount of daemon thread time, which may cause the daemon thread to not respond to other processes in a timely manner. The maximum number of bytes that can be written to a memory block in the DataSyn system is 1024 bytes. If the byte limit is increased, more data can be written, but the efficiency will also be reduced.
[0140] In order to solve the problem of low efficiency caused by frequent and large-scale communications, two methods are provided.
[0141] In the first method, two processes can first register and write their own information (such as IP address, listening port number), and then the two processes can query each other's information and use sockets for point-to-point communication. This method can avoid the reduction of DataSyn system efficiency and real-time performance, while also meeting the needs.
[0142] The second method is to create one or more independent or related DataSyn systems for multiple processes that need frequent large-scale data communication, and let different process groups use different DataSyn systems, as long as resource conflicts such as listening ports and communication frame encapsulation protocol numbers are avoided. Large-scale data communication requires the opening of dedicated shared memory, and the DataSyn system still only transmits data not exceeding 1024 bytes. However, as long as the data not exceeding 1024 bytes contains an offset pointer, the communicating parties can calculate and find the correct address from the dedicated shared memory in an agreed manner, write and read large blocks of data, and break through the 1024-byte content limit.
[0143] Please refer to Fig. 9 , which is a schematic diagram of multiple DataSyn systems provided in an embodiment of the present application. In a device or a DataSyn system composed of multiple devices in an FTTR network, there are two DataSyn systems, wherein, if it is a PON device or a WIFI router, shared memory area 1 and shared memory area 2 both include a public shared area and a shared memory area of this device, and if it is an FTTR device, shared memory area 1 and shared memory area 2 both include a public shared area, a shared memory area of this device, and a shared memory area of other devices. The conventional process group DataSyn system on the left works in a conventional manner, and the high-frequency and large-volume process group DataSyn system on the right serves the high-frequency and large-volume communication process group specifically. Shared memory area 2 stores a large number of offset pointers pointing to shared memory area 3, and shared memory area 3 is a shared memory dedicated to the high-frequency and large-volume process group.
[0144] The embodiment of the present application fully utilizes the advantages of CPU multi-core and gives full play to the high efficiency of the DataSyn system to meet the different needs of different process groups. Different process groups do not affect each other, and the high frequency and large data volume reading and writing of a certain process group will not affect the communication efficiency of other process groups.
[0145] In a possible embodiment, a unified set of standard API interfaces for the DataSyn system is designed as follows:
[0146] int dataSynQuery(int devNo, const char * key, char * value);
[0147] int dataSynUpdateKey(int devNo, const char * key, char * value, intlen);
[0148] int dataSynSubscribeKey(int devNo, const char * key);
[0149] The description of each function is shown in Table 4:
[0150] Table 4
[0151]
[0152] The common parameters are shown in Table 5:
[0153] Table 5
[0154]
[0155] Here we only list three most basic APIs, which have few parameters, clear meanings, and are easy to use, but they have covered most of the functions in the DataSyn system. By adding some more APIs and expanding the parameters, we can fully cover all the functions of DataSyn (including the coexistence of multiple DataSyns). The concise API design ensures its ease of use and functional scalability.
[0156] Please refer to Fig.10, which is a schematic diagram of the overall architecture of the DataSyn system provided in the embodiment of the present application. There are multiple processes in the user space. Process 1 in the user space can use the dataSynUpdateKey function system call to write data to the daemon thread. The daemon thread writes the data to the shared memory area of the device. Process 2 in the user space can use the dataSynQuery function to read data directly from the kernel space. It is also possible to start a data synchronization thread between devices to send and receive synchronization data packets with other devices.
[0157] In summary, the embodiment of the present application provides a data synchronization method based on FTTR / PON / WIFI, which is applied to the DataSyn system. The user space process can subscribe to the key of interest. When the value of the key (offset address) changes, the registered process is notified to realize the unique subscription and publishing mechanism of the DataSyn system. The DataSyn system also designs a set of general and concise interfaces such as reading and writing shared memory, subscription and publishing, and provides them for each process to call in the form of a dynamic library. For write operations, the user space process wakes up the DataSyn daemon thread through a system call, and the daemon thread is responsible for updating the received data to the corresponding position of the shared memory, ensuring the consistency and real-time nature of data synchronization. For read operations, the user space process can query the specific location of the data through the specified key, read and copy the data to its own process space, and realize efficient data sharing.
[0158] The DataSyn system provides an efficient, secure and transparent mechanism that enables different user space processes on the same or different devices to seamlessly share and synchronize data without having to worry about the overhead and complexity of traditional inter-process communication mechanisms. It is suitable for scenarios with high requirements for data real-time performance, such as real-time data processing and high-speed data exchange, and can greatly improve the overall performance and response speed of the system.
[0159] The DataSyn system can also make full use of the high-speed optical fiber connection between FTTR devices to create an inter-device data synchronization thread, which is specifically used for real-time synchronization of data in shared memory between devices, so that each process can query not only the shared data of all processes on the same device, but also the shared data of all processes on different devices. The inter-device synchronization thread runs in the Linux kernel and uses the driver layer interface to directly send and receive packets to synchronize data without going through the TCP / IP protocol stack, which is more efficient and concise.
[0160] The DataSyn system fully considers and solves potential problems such as memory management, security, and concurrency control, ensuring system stability and data consistency. In addition, the design of the DataSyn module also allows for flexible expansion and adjustment to meet the needs of different application scenarios, providing strong support for building high-performance systems.
[0161] Based on the same inventive concept, Fig.11 As shown, the embodiment of the present application also provides a data synchronization device based on FTTR / PON / WIFI, which is arranged in any device of FTTR / PON / WIFI; the shared memory of any device includes a public shared memory area and a shared memory area of the device; the public shared memory area pre-stores a plurality of fixed keys; the shared memory area of the device is used to store data of any device; the device includes:
[0162] A receiving module is used to start a daemon thread and receive a data write request initiated by the first process; the data write request is used to request to write target data;
[0163] A calculation module, used for calculating the target key corresponding to the target data by using a minimum perfect hash algorithm to obtain a first offset address;
[0164] A writing module, used for, if the key pointed to by the first memory address in the public shared memory area is the same as the target key, querying the second offset address corresponding to the target key, and writing the target data into the shared memory area of the device according to the second memory address; the first memory address is the sum of the first address of the public shared memory and the first offset address; the second memory address is the sum of the first address of the shared memory area of the device and the second offset address;
[0165] A sending module, used to send a UDP data packet to a second process subscribed to the target key; the UDP data packet is used to indicate that the offset address corresponding to the target key is updated;
[0166] The reading module is used to read the target data from the shared memory area of the device in response to the query request of the second process.
[0167] It should be noted that each module in the data synchronization device based on FTTR / PON / WIFI in this embodiment corresponds one-to-one to each step in the data synchronization method based on FTTR / PON / WIFI in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned data synchronization method based on FTTR / PON / WIFI, which will not be repeated here.
[0168] Based on the same inventive concept, the present application also provides a computer device, which is equivalent to an IPTV remote control. The computer device includes a processor, a memory, and a computer program stored in the memory. When the computer program is executed by the processor, the aforementioned FTTR / PON / WIFI-based data synchronization method is implemented.
[0169] Based on the same inventive concept, the present application also provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the aforementioned data synchronization method based on FTTR / PON / WIFI is implemented.
[0170] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.
[0171] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.
[0172] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).
[0173] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.
[0174] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0175] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0176] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0177] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A data synchronization method based on FTTR / PON / WIFI, characterized in that: Applicable to any device of FTTR / PON / WIFI; the shared memory of any device includes a public shared memory area and a local device shared memory area; the public shared memory area pre-stores a plurality of fixed keys; The shared memory area of the device stores data according to a preset data structure; the data structure uses a version number, a timestamp, and a data length as a data header, and the data itself as a data body; The initial value of the version number is 0; The method comprises: Starting a daemon thread to receive a data write request initiated by the first process; the data write request is used to request to write target data; Using a minimum perfect hash algorithm to operate on a target key corresponding to the target data to obtain a first offset address; If the key pointed to by the first memory address in the public shared memory area is the same as the target key, the second offset address corresponding to the target key is queried, and the target data is written into the shared memory area of the local device according to the second memory address; the first memory address is the sum of the first address of the public shared memory area and the first offset address; the second memory address is the sum of the first address of the shared memory area of the local device and the second offset address; Sending a UDP data packet to a second process that has subscribed to the target key; the UDP data packet is used to indicate that the data pointed to by the offset address corresponding to the target key is updated; In response to the query request of the second process, reading the target data from the shared memory area of the local device; Before sending the UDP data packet to the second process subscribed to the target key, the method further comprises: receiving a registration request from the second process; the registration request comprises port information of a listening port and at least one key to be subscribed; the at least one key comprises a target key; the listening port is an idle UDP port applied by the second process to the Linux system; registering the second process according to the at least one key and the port information of the listening port; The sending of the UDP data packet to the second process subscribed to the target key comprises: taking the listening port as the destination port and the IP of the device where the second process is located as the destination IP, constructing a UDP data packet, and sending the UDP data packet to the second process; Writing the target data into the shared memory area of the local device according to the second memory address comprises: obtaining a third offset address according to the second memory address, adding the first address of the shared memory area of the local device to the third offset address to obtain a third memory address; querying the original data from the shared memory area of the local device according to the third memory address and the data length corresponding to the target key; judging whether the target data is different from the original data; if so, releasing the memory pointed to by the third memory address, applying for new memory according to the length of the target data, adding 1 to the version number, updating the timestamp and the data length, and after writing the target data into the new memory, adding 1 to the version number again; The step of responding to the query request of the second process and reading the target data from the shared memory area of the local device comprises: receiving and parsing the query request of the second process to obtain the target key; the query request includes the target key; obtaining the version number according to the target key; if the version number is an odd number, continuing to wait; if the version number is an even number, reading the target data from the shared memory area of the local device; Any one of the devices is a device of the FTTR, and the shared memory of any one of the devices includes a copy data area; after writing the target data into the shared memory area of the device according to the second memory address, the method further includes: storing the target key, the length of the target data, and the updated offset address in a synchronization linked list; starting an inter-device data synchronization thread, and if the length of the synchronization linked list is greater than a preset threshold, obtaining a partition lock of any one of the devices, copying all data in the shared memory area of the device to the copy data area; releasing the partition lock, and sending the data in the copy data area to other devices of the FTTR.
2. The data synchronization method based on FTTR / PON / WIFI according to claim 1, characterized in that: The step of sending the data in the copied data area to other devices in the FTTR comprises: Encapsulating the data in the copy data area according to a preset packet format to obtain encapsulated data; the packet format includes a destination MAC address, a source MAC address, a protocol number, a packet type, a partition number, and a payload block; The driver layer is called to send the encapsulated data to other devices of the FTTR.
3. The data synchronization method based on FTTR / PON / WIFI according to claim 1, characterized in that: The step of starting a synchronization data thread between devices includes: If the frequency of writing data within the preset time period is greater than or equal to the preset frequency, the inter-device data synchronization thread is started regularly; If the frequency of writing data within the preset time period is less than the preset frequency, the inter-device data synchronization thread is immediately started.
4. A data synchronization device based on FTTR / PON / WIFI, characterized in that: The device is arranged in any device of FTTR / PON / WIFI; the shared memory of any device comprises a public shared memory area and a shared memory area of the device; the public shared memory area stores a plurality of fixed keys in advance; The shared memory area of the device stores data according to a preset data structure; the data structure uses a version number, a timestamp, and a data length as a data header, and the data itself as a data body; The initial value of the version number is 0; The device comprises: A receiving module, used to start a daemon thread and receive a data write request initiated by a first process; the data write request is used to request to write target data; A calculation module, used for calculating the target key corresponding to the target data by using a minimum perfect hash algorithm to obtain a first offset address; A writing module, used for, if the key pointed to by the first memory address in the public shared memory area is the same as the target key, querying the second offset address corresponding to the target key, and writing the target data into the shared memory area of the local device according to the second memory address; the first memory address is the sum of the first address of the public shared memory area and the first offset address; the second memory address is the sum of the first address of the shared memory area of the local device and the second offset address; A sending module, used to send a UDP data packet to a second process subscribed to the target key; the UDP data packet is used to indicate that the data pointed to by the offset address corresponding to the target key is updated; A reading module, configured to read the target data from the shared memory area of the local device in response to a query request of the second process; Before sending a UDP data packet to the second process subscribed to the target key, the method further includes: receiving a registration request from the second process; the registration request includes port information of a listening port and at least one key to be subscribed; the at least one key includes a target key; the listening port is an idle UDP port applied by the second process to the Linux system; registering the second process according to the at least one key and the port information of the listening port; The sending of the UDP data packet to the second process subscribed to the target key comprises: taking the listening port as the destination port and the IP of the device where the second process is located as the destination IP, constructing a UDP data packet, and sending the UDP data packet to the second process; Writing the target data into the shared memory area of the local device according to the second memory address comprises: obtaining a third offset address according to the second memory address, adding the first address of the shared memory area of the local device to the third offset address to obtain a third memory address; querying the original data from the shared memory area of the local device according to the third memory address and the data length corresponding to the target key; judging whether the target data is different from the original data; if so, releasing the memory pointed to by the third memory address, applying for new memory according to the length of the target data, adding 1 to the version number, updating the timestamp and the data length, and after writing the target data into the new memory, adding 1 to the version number again; The step of responding to the query request of the second process and reading the target data from the shared memory area of the local device comprises: receiving and parsing the query request of the second process to obtain the target key; the query request includes the target key; obtaining the version number according to the target key; if the version number is an odd number, continuing to wait; if the version number is an even number, reading the target data from the shared memory area of the local device; Any one of the devices is a device of the FTTR, and the shared memory of any one of the devices includes a copy data area; after writing the target data into the shared memory area of the device according to the second memory address, it also includes: storing the target key, the length of the target data, and the updated offset address in a synchronization linked list; starting an inter-device data synchronization thread, if the length of the synchronization linked list is greater than a preset threshold, obtaining a partition lock of any one of the devices, copying all data in the shared memory area of the device to the copy data area; releasing the partition lock, and sending the data in the copy data area to other devices of the FTTR.
5. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the FTTR / PON / WIFI-based data synchronization method according to any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the FTTR / PON / WIFI-based data synchronization method according to any one of claims 1 to 3.
