File synchronization method, system and device, medium and product

Abstract

The invention belongs to the technical field of industrial control system operation security, and discloses a file synchronization method, system and device, a medium and a product, and the method comprises the steps: detecting the update state of a target file in a current upper computer; writing the path of the target file and the updating operation state into a preset file sharing memory; loading the current file in the path of the target file into a preset file sharing memory; reading an update operation state of a target file in the current upper computer and a current file under a path of the target file, and uploading the update operation state and the current file to a file server for storage; issuing the current file to a to-be-synchronized target upper computer by adopting an incremental file synchronization technology or a self-adaptive synchronization strategy technology of content identification; according to the method, efficient, reliable and flexible synchronization of the files among the upper computers in the industrial control system is achieved, manual operation is not needed, the risk of human errors or omission is avoided, and file consistency and data accuracy among the upper computer devices in the industrial control system are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial control system operation safety, and in particular relates to a file synchronization method, system, equipment, medium and product. Background Art

[0002] In the field of wind power, industrial control systems are used to control and monitor the operation of wind turbines to ensure their safe operation. During the operation of the industrial control system, various files of the industrial control system need to be synchronized between various host computers. For example, parameter configuration files, database configuration files and screen files are synchronized between the history station, engineer station and operator station to ensure the consistency of working status and data accuracy between various host computers in the industrial control system.

[0003] Due to the operation needs of the wind power system, it is usually necessary to update certain files such as adding, changing or deleting. At this time, the files that have been updated need to be synchronized. At present, the traditional file synchronization method is usually manually updated by operators for each host computer. However, omissions or errors are very likely to occur during manual updates, resulting in inconsistent working states between various devices, which seriously threatens the safe operation of the industrial control system. Summary of the invention

[0004] In response to the technical problems existing in the prior art, the present invention provides a file synchronization method, system, device, medium and product to solve the technical problem that traditional file synchronization methods are prone to omissions or errors, resulting in inconsistent working states between various devices, which in turn seriously threatens the safe operation of industrial control systems.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] The present invention provides a file synchronization method, comprising:

[0007] Check the update status of the target file in the current host computer;

[0008] If the update status of the target file in the current host computer changes, the path and update operation status of the target file in the current host computer are written into the preset file shared memory;

[0009] According to the path of the target file in the current host computer written in the preset file sharing memory, the current file under the path of the target file in the current host computer is loaded into the preset file sharing memory;

[0010] Read the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer from the preset file sharing memory, and upload them to the file server for storage;

[0011] According to the update operation status of the target file in the current host computer stored in the file server, the current file under the path of the target file in the current host computer stored in the file server is sent to the target host computer to be synchronized using content-aware incremental file synchronization technology or adaptive synchronization strategy technology.

[0012] Furthermore, the process of detecting the update status of the target file in the current host computer is as follows:

[0013] Using the file monitoring process, read the monitoring path pre-specified in the monitorConfig.xml file under the / users / ems / fileServer path in the current host computer, and obtain the path of the target file in the current host computer;

[0014] According to the number of paths of the target files in the current host computer, update status detection threads are created respectively; wherein, in each update status detection thread, the FileSystemWatcher control is used to detect the file update status under the path of the target file in the current host computer.

[0015] Furthermore, if the update operation status of the target file in the current host computer is a change operation, the current file under the path of the target file in the current host computer stored in the file server is sent to the target host computer to be synchronized using the content-aware incremental file synchronization technology;

[0016] The process of using content-aware incremental file synchronization technology to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized is as follows:

[0017] Using a preset content partitioning algorithm, the current file under the path of the target file in the current host computer and the file to be synchronized in the target host computer to be synchronized are divided into blocks respectively, and the source file block result and the file to be synchronized block result are obtained;

[0018] Calculate the content hash values ​​of the source file segmentation result and the to-be-synchronized file segmentation result respectively, and obtain the source file content hash table and the to-be-synchronized file content hash table;

[0019] Compare the source file content hash table with the to-be-synchronized file content hash table, filter out content blocks that differ between the source file segmentation result and the to-be-synchronized file segmentation result, and obtain the difference content blocks;

[0020] The difference content blocks in the block result of the source file are replaced with the difference content blocks in the block result of the file to be synchronized, so as to obtain the update result of the file to be synchronized;

[0021] Calculate the overall content hash value of the update result of the current file and the file to be synchronized under the path of the target file in the current host computer respectively, and confirm the consistency.

[0022] Furthermore, if the update operation status of the target file in the current host computer is a deletion or addition operation, the current file under the path of the target file in the current host computer stored in the file server is sent to the target host computer to be synchronized using the adaptive synchronization strategy technology;

[0023] The process of using the adaptive synchronization strategy technology to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized is as follows:

[0024] Obtain the current network bandwidth of the primary network link and the backup network link between the file server and the target host computer to be synchronized;

[0025] If the current network bandwidth of at least one network link among the primary network link and the backup network link is lower than the preset bandwidth threshold, the network link with the lower current network bandwidth is selected to establish the link, and the current file under the path of the target file in the current host computer is transmitted to the target host computer to be synchronized, and the current file under the path of the target file in the current host computer is used to replace the to-be-synchronized file of the target host computer to be synchronized as a whole;

[0026] If the current network bandwidths of the primary network link and the backup network link are both higher than the preset bandwidth threshold, links are established simultaneously through the primary network link and the backup network link, and the current file under the path of the target file in the current host computer is transmitted to the target host computer to be synchronized using the preset flow control algorithm, and the current file under the path of the target file in the current host computer is used to replace the to-be-synchronized file of the target host computer to be synchronized as a whole.

[0027] Furthermore, in the process of sending the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized, a distributed file update strategy is used to perform file transmission.

[0028] Furthermore, the process of using a distributed file update strategy to transfer files is as follows:

[0029] Get the CPU usage and memory usage of the primary file server of the file server;

[0030] If the CPU occupancy rate or memory occupancy rate of the primary file server exceeds a preset occupancy rate threshold, the primary file server and the backup file server of the file server are used to perform file transfer simultaneously; wherein the primary file server is allocated less file transfer tasks than the backup file server.

[0031] The present invention also provides a file synchronization system, comprising:

[0032] The update status detection module is used to detect the update status of the target file in the current host computer;

[0033] The shared memory writing module is used to write the path and update operation status of the target file in the current host computer into the preset file shared memory if the update status of the target file in the current host computer changes;

[0034] A file loading module is used to load the current file under the path of the target file in the current host computer into the preset file sharing memory according to the path of the target file in the current host computer written in the preset file sharing memory;

[0035] A shared memory reading module is used to read the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer from the preset file shared memory, and upload them to the file server for storage;

[0036] The transmission synchronization module is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized according to the update operation status of the target file in the current host computer stored in the file server, using content-aware incremental file synchronization technology or adaptive synchronization strategy technology.

[0037] The present invention also provides a file synchronization device, comprising:

[0038] a processor suitable for executing a computer program;

[0039] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the file synchronization method is executed.

[0040] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the file synchronization method is implemented.

[0041] The present invention also provides a computer program product, characterized in that the computer program product comprises a computer program, and the computer program implements the file synchronization method when executed by a processor.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention provides a file synchronization method, which detects the update status of a target file in a current host computer in real time, writes the changed current file into a file shared memory, reads the current file in the file shared memory and uploads it to a file server for storage; when file synchronization is required, based on the update operation status of the target file, content-recognized incremental file synchronization technology and adaptive synchronization strategy technology are used to synchronize the current file stored in the file server to the target host computer to be synchronized, thereby achieving efficient, reliable and flexible synchronization of files between various host computers in an industrial control system, without the need for manual operation, avoiding the risk of human error or omission, effectively ensuring the file consistency and data accuracy between various host computer devices in the industrial control system, and thus ensuring the safe operation of the industrial control system.

[0044] Furthermore, the FileSystemWatcher control detects the file update status under the path of the target file in the current host computer, realizes real-time detection of the update status of the target file in the current host computer, ensures the consistency and accuracy of files and data between the host computers in the industrial control system, and avoids the risk of data delay or loss.

[0045] Furthermore, the incremental file synchronization technology based on content recognition is used to synchronize the target file that has undergone the change operation, so as to synchronize the changed content blocks in the target file without synchronizing the entire file, thereby effectively improving the file synchronization efficiency.

[0046] Furthermore, according to the current network bandwidth of the primary network link and the backup network link, the primary network link or the backup network link is selected for file transmission, which can reduce the occupation of network resources and ensure the efficiency and real-time performance of file synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 This is a flowchart of the file synchronization method described in Example 1;

[0049] Figure 2 This is a structural block diagram of the file synchronization system described in Example 2;

[0050] Figure 3 This is a structural block diagram of the file synchronization device described in Example 3. DETAILED DESCRIPTION

[0051] In order to make the technical problems, technical solutions and beneficial effects solved by this application clearer, the technical solutions in the embodiments of this application will be described clearly and completely in combination with the drawings in the embodiments of this application; obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] Example 1

[0053] Take the file synchronization process between the host computers in a wind power industrial control system as an example; Figure 1 As shown, this embodiment 1 provides a file synchronization method, including the following steps:

[0054] Step 1: Call the log recording function interface of the industrial control system, and use the log recording function interface to record the log information of subsequent file synchronization upload and download operations.

[0055] Step 2: Use the file synchronization management process to start the file synchronization service process; wherein the file synchronization service process includes a synchronization heartbeat daemon process, a file monitoring process, and a file update process. Specifically, the startup process has the following steps:

[0056] Step 21: Start the file synchronization management process.

[0057] Step 22: After the file synchronization management process is started, the file synchronization management process is used to start the synchronization heartbeat daemon process, the file monitoring process and the file update process.

[0058] Step 23, using the file synchronization management process, the application process name and process PID number of the started synchronization mentality daemon process, file monitoring process and file update process are recorded in a preset file synchronization dedicated database; wherein the application process name and process PID number are stored in the file synchronization list of the preset file synchronization dedicated database.

[0059] It should be noted that when a process in the file synchronization service process fails to start, the file synchronization management process is used to restart the failed process; if the startup fails three times in a row, the startup error log is printed and an alarm message of process startup failure is sent.

[0060] Step 3: Process running status monitoring. The process of monitoring the process running status is as follows:

[0061] Step 31: Use the file synchronization management process to perform a real-time query on the application process name stored in the file synchronization list of the preset file synchronization dedicated database to obtain a name query result.

[0062] Step 32: Use the process PID query method provided by ACE to query the process PID number stored in the file synchronization list of the preset file synchronization dedicated database to obtain the PID query result.

[0063] Step 33: According to the name query result and the PID query result, determine whether the synchronous heartbeat daemon process, the file monitoring process and the file update process are in a running state.

[0064] Step 34: If a process in the file synchronization service process is not in a running state, the process startup method provided by ACE is used to start the unstarted process, and the restarted application process name and process PID number are recorded in the preset file synchronization dedicated database.

[0065] Step 4: Identify abnormal processes. The identification process of abnormal processes is as follows:

[0066] Step 41, using the file synchronization management process, periodically query the CPU and memory occupancy of the file synchronization service process to obtain the process CPU occupancy rate and the process memory occupancy rate; using the application process name as an index, store the process CPU occupancy rate and the process memory occupancy rate in the file synchronization list of the preset file synchronization dedicated database.

[0067] Step 42: Utilize the file synchronization management process to obtain the process CPU occupancy rate and process memory occupancy rate of a process in the file synchronization service process at the current moment; and filter the process CPU occupancy rate and process memory occupancy rate of the corresponding process at the previous moment from the file synchronization list according to the process PID number.

[0068] Step 43: Compare the CPU occupancy rate and the memory occupancy rate of the process at the current moment with the CPU occupancy rate and the memory occupancy rate of the process at the previous moment to obtain an occupancy comparison result; if the occupancy comparison result exceeds a preset occupancy change threshold, the process is abnormal, and a function is called to inflate the abnormal process.

[0069] Step 5: Use the file synchronization management process to create a device physical information monitoring thread, and use the device physical information monitoring thread to detect the physical device information of the current host computer in real time. The physical device information of the current host computer includes the memory usage, CPU usage, network uplink rate and network downlink rate of the current host computer.

[0070] Step 6: Establish a link between the host computer and the file server.

[0071] The file synchronization heartbeat daemon is used to read the host name of the file server in the industrial control system and the host name of the current host computer; the host name of the file server in the industrial control system is compared with the host name of the current host computer to determine whether the current host computer is a file server; if so, the Server service in the file synchronization heartbeat daemon is started, and other host computers in the industrial control system are waiting to establish a link with the file server. The host name of the file server in the industrial control system is stored in the configuration file config.xml under the path of the host computer / users / ems / fileServer, and the file server includes a redundantly set primary file server and a backup file server.

[0072] Step 7: Use the file synchronization heartbeat daemon to maintain the on / off status of the primary network link and the backup network link.

[0073] Among them, the main network link is the main link between the file server and other host computers in the industrial control system, that is, network A; the backup network link is the backup link between the file server and the target host computer to be synchronized, that is, network B.

[0074] Specifically, the process of maintaining the on / off status of the primary network link and the backup network link is as follows:

[0075] If the current host computer is a file server, the A network heartbeat daemon Server service and the B network heartbeat daemon Server service in the file synchronization heartbeat daemon process are started; wait for other hosts to start the Client service in the file synchronization heartbeat daemon process, establish links with the A network and the B network of the primary file server in the file server, and send heartbeat messages; the primary file server receives the heartbeat message and replies to determine whether the primary file server is connected or disconnected; if the A network and the B network of the primary file server cannot be linked or there is no heartbeat reply, it will automatically switch to the backup file server in the file server.

[0076] Step 8: Use the file synchronization heartbeat daemon to open up and initialize three shared memories in the memory; wherein the three shared memories include information shared memory, message shared memory and file shared memory.

[0077] Specifically, the message shared memory is used to store the device information of all host computers in the industrial control system and is recorded as shared memory 1; wherein the device information of the host computer includes the host computer host name, A network IP, B network IP, CPU occupancy rate, memory occupancy rate, A network uplink and downlink bandwidth and B network uplink and downlink bandwidth;

[0078] Specifically, the message shared memory is used to store communication messages between file synchronization service processes and is recorded as shared memory 2; wherein, the communication process between file synchronization service processes is specifically as follows: all communication methods between file synchronization service processes are sent to the message shared memory, and the message distribution function in the file synchronization heartbeat daemon process is used to distribute the received message to the target process by detecting the unique identification number carried in the received message, sending the message to the message queue of the target process, and notifying the target process to process the received message.

[0079] Specifically, the file shared memory is used to store the path of the target file in the current host computer, the update operation status and the current file mapping under the path of the target file in the current host computer, and is recorded as shared file 3; taking the history station as an example, after the history station receives the updated file, the path and number of the file to be updated are written into the file shared memory; at the same time, the file to be synchronized is mapped to the file shared memory.

[0080] Step 9: Detect the update status of the target file in the current host computer. The detection process is as follows:

[0081] Utilize the file monitoring process to read the monitoring path pre-specified in the monitorConfig.xml file under the / users / ems / fileServer path in the current host computer, and obtain the path of the target file in the current host computer; create update status detection threads respectively according to the number of paths of the target files in the current host computer; wherein, in each update status detection thread, use the FileSystemWatcher control to detect the file update status under the path of the target file in the current host computer.

[0082] Step 10: If the update status of the target file in the current host computer changes, the path and update operation status of the target file in the current host computer are written into the preset file shared memory. The update operation status of the target file in the current host computer includes a change operation, a delete operation or a new operation; at the same time, the file update process is notified through the message shared memory to perform an update operation on the file whose update status has changed.

[0083] Step 11: according to the path of the target file in the current host computer written in the preset file sharing memory, the current file under the path of the target file in the previous host computer is loaded into the preset file sharing memory.

[0084] Specifically, after the file update process obtains the notification of file update through the message shared memory, the file update process obtains the path and update operation status of the target file in the current host computer from the file shared memory; if there is a file to be synchronized, the current file under the path of the target file in the previous host computer is loaded into the preset file shared memory.

[0085] Step 12, create an upload thread; use the upload thread to read the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer from the preset file sharing memory, and upload them to the file server for storage.

[0086] It should be noted that the main file server and the standby file server in the file server synchronously store the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer.

[0087] Step 13: According to the update operation status of the target file in the current host computer stored in the file server, content-aware incremental file synchronization technology or adaptive synchronization strategy technology is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized.

[0088] Specifically, if the update operation status of the target file in the current host computer is a change operation, the content-aware incremental file synchronization technology is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized; if the update operation status of the target file in the current host computer is a deletion or addition operation, the adaptive synchronization strategy technology is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized.

[0089] It should be noted that the process of using the content-aware incremental file synchronization technology to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized is as follows:

[0090] Step 1301, using a preset content partitioning algorithm, the current file under the path of the target file in the current host computer and the file to be synchronized in the target host computer to be synchronized are divided into blocks respectively to obtain the source file partitioning result and the file to be synchronized partitioning result; wherein, the preset content partitioning algorithm adopts a block-level hash algorithm or a sliding window hash method.

[0091] Step 1302: Calculate the content hash values ​​of the source file segmentation result and the to-be-synchronized file segmentation result respectively, and obtain the source file content hash table and the to-be-synchronized file content hash table.

[0092] Step 1303: Compare the source file content hash table with the to-be-synchronized file content hash table, filter out content blocks that differ between the source file segmentation result and the to-be-synchronized file segmentation result, and obtain difference content blocks.

[0093] Step 1304: Replace the difference content blocks in the block result of the file to be synchronized with the difference content blocks in the block result of the file to be synchronized, and obtain an update result of the file to be synchronized.

[0094] Step 1305: Calculate the overall content hash values ​​of the update results of the current file and the file to be synchronized under the path of the target file in the current host computer, and confirm the consistency.

[0095] It should be noted that the process of using the adaptive synchronization strategy technology to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized is as follows:

[0096] Step 1311: using the upload thread, obtain the current network bandwidth of the primary network link and the backup network link between the file server and the target host computer to be synchronized from the information shared memory.

[0097] Step 1312: If the current network bandwidth of at least one network link among the primary network link and the backup network link is lower than the preset bandwidth threshold, the network link with the lower current network bandwidth is selected to establish the link, and the current file under the path of the target file in the current host computer is transmitted to the target host computer to be synchronized, and the current file under the path of the target file in the current host computer is used to replace the to-be-synchronized file of the target host computer to be synchronized as a whole;

[0098] Step 1313: If the current network bandwidths of the primary network link and the backup network link are both higher than the preset bandwidth threshold, links are established simultaneously through the primary network link and the backup network link, and the current file under the path of the target file in the current host computer is transmitted to the target host computer to be synchronized using the preset flow control algorithm, and the current file under the path of the target file in the current host computer is used to replace the to-be-synchronized file of the target host computer to be synchronized as a whole; wherein the preset flow control algorithm includes a token bucket algorithm, a leaky bucket algorithm or a congestion control algorithm.

[0099] In this embodiment 1, in the process of sending the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized, a distributed file update strategy is used for file transmission; wherein, the process of using the distributed file update strategy for file transmission is specifically as follows:

[0100] Obtain the CPU occupancy rate and memory occupancy rate of the primary file server of the file server; if the CPU occupancy rate or memory occupancy rate of the primary file server exceeds a preset occupancy rate threshold, use the primary file server and the backup file server of the file server to simultaneously perform file transfer; wherein, the primary file server is allocated less file transfer tasks than the backup file server.

[0101] Step 14: When an external process stop command is received, the file synchronization service process is stopped, and the data in the three shared memories are destroyed, thereby terminating the file synchronization method.

[0102] The file synchronization method described in Example 1 can quickly synchronize the files of the host computer of the industrial control system, ensure the consistency of the working status and data accuracy between the devices, and ensure the safe operation of the industrial control system; specifically, the file synchronization management process monitors the file synchronization heartbeat daemon process, the file monitoring process, the file update process, and the CPU, memory and network bandwidth parameters of the entire host computer to prevent the host computer from being stuck due to a high CPU occupancy rate caused by an abnormal process, thereby ensuring the smooth operation of the host computer file synchronization system; the file synchronization heartbeat daemon process manages and maintains three shared memories, uniformly manages the host name, A network IP and B network IP information of all host computers that start the file synchronization system, and uses shared memory to complete communication and file transfer between processes, thereby accelerating the file transfer rate and improving the synchronization efficiency of the file synchronization system; at the same time, the file update process uses different incremental file technologies of content recognition, adaptive synchronization strategy technology, distributed file update strategy and other methods according to factors such as the CPU, memory, and network bandwidth of the primary and standby file servers to quickly, efficiently and stably complete the file synchronization of the industrial control system.

[0103] Example 2

[0104] As attached Figure 2 As shown, this embodiment 2 provides a file synchronization system, including: a log module, a process startup module, a process status monitoring module, an abnormal process identification module, a device information monitoring module, a link establishment module, a link maintenance module, a shared memory module, an update status detection module, a shared memory write module, a file loading module, a shared memory read module, a transmission synchronization module and an end module.

[0105] A log module is used to call the log recording function interface of the industrial control system, and use the log recording function interface to record the log information of subsequent file synchronization upload and download operations; a process startup module is used to use the file synchronization management process to start the file synchronization service process; wherein the file synchronization service process includes a synchronization heartbeat daemon process, a file monitoring process and a file update process; a process status monitoring module is used to monitor the process running status; an abnormal process identification module is used to identify abnormal processes; a device information monitoring module is used to use the file synchronization management process to create a device physical information monitoring thread, and use the device physical information monitoring thread to detect the physical device information of the current host computer in real time; a link building module is used to build a link between the host computer and the file server; a link maintenance module is used to use the file synchronization heartbeat daemon process to maintain the on-off status of the main network link and the backup network link; a shared memory module is used to use the file synchronization heartbeat daemon process to open up and initialize three shared memories in the memory; wherein the three shared memories include information shared memory, message shared memory and file shared memory; an update status detection module is used to detect the update status of the target file in the current host computer; A shared memory writing module is used to write the path and update operation status of the target file in the current host computer into the preset file shared memory if the update status of the target file in the current host computer changes; a file loading module is used to load the current file under the path of the target file in the previous host computer into the preset file shared memory according to the path of the target file in the current host computer written in the preset file shared memory; a shared memory reading module is used to create an upload thread; using the upload thread, the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer are read from the preset file shared memory, and uploaded to the file server for storage; a transmission synchronization module is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized by using the content-recognized incremental file synchronization technology or the adaptive synchronization strategy technology according to the update operation status of the target file in the current host computer stored in the file server; an end module is used to stop the file synchronization service process when an external process stop command is received, and destroy the data in the three shared memories to end the file synchronization method.

[0106] Example 3

[0107] As attached Figure 3 As shown, this embodiment 3 also provides a file synchronization device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a file synchronization program; wherein the attached Figure 3 The communication interface in is used to access external devices to obtain data.

[0108] When the processor executes the computer program, the steps in the above-mentioned file synchronization method are implemented, for example:

[0109] Call the log recording function interface of the industrial control system, and use the log recording function interface to record the log information of subsequent file synchronization upload and download operations; use the file synchronization management process to start the file synchronization service process; wherein, the file synchronization service process includes a synchronization heartbeat daemon process, a file monitoring process and a file update process; process running status monitoring; abnormal process identification; use the file synchronization management process to create a device physical information monitoring thread, and use the device physical information monitoring thread to detect the physical device information of the current host computer in real time; establish a link between the host computer and the file server; use the file synchronization heartbeat daemon process to maintain the on-off status of the main network link and the backup network link; use the file synchronization heartbeat daemon process to open up and initialize three shared memories in the memory; wherein, the three shared memories include information shared memory, message shared memory and file shared memory; detect the update status of the target file in the current host computer; if the update status of the target file in the current host computer changes , then write the path and update operation status of the target file in the current host computer into the preset file shared memory; according to the path of the target file in the current host computer written in the preset file shared memory, load the current file under the path of the target file in the previous host computer into the preset file shared memory; create an upload thread; use the upload thread to read the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer from the preset file shared memory, and upload them to the file server for storage; according to the update operation status of the target file in the current host computer stored in the file server, use the content-recognized incremental file synchronization technology or the adaptive synchronization strategy technology to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized; when an external process stop command is received, stop the file synchronization service process, destroy the data in the three shared memories, and end the file synchronization method.

[0110] Alternatively, when the processor executes the computer program, the functions of each module in the above-mentioned file synchronization system are realized, for example:

[0111] A log module is used to call the log recording function interface of the industrial control system, and use the log recording function interface to record the log information of subsequent file synchronization upload and download operations; a process startup module is used to use the file synchronization management process to start the file synchronization service process; wherein the file synchronization service process includes a synchronization heartbeat daemon process, a file monitoring process and a file update process; a process status monitoring module is used to monitor the process running status; an abnormal process identification module is used to identify abnormal processes; a device information monitoring module is used to use the file synchronization management process to create a device physical information monitoring thread, and use the device physical information monitoring thread to detect the physical device information of the current host computer in real time; a link building module is used to build a link between the host computer and the file server; a link maintenance module is used to use the file synchronization heartbeat daemon process to maintain the on-off status of the main network link and the backup network link; a shared memory module is used to use the file synchronization heartbeat daemon process to open up and initialize three shared memories in the memory; wherein the three shared memories include information shared memory, message shared memory and file shared memory; an update status detection module is used to detect the update status of the target file in the current host computer; A shared memory writing module is used to write the path and update operation status of the target file in the current host computer into the preset file shared memory if the update status of the target file in the current host computer changes; a file loading module is used to load the current file under the path of the target file in the previous host computer into the preset file shared memory according to the path of the target file in the current host computer written in the preset file shared memory; a shared memory reading module is used to create an upload thread; using the upload thread, the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer are read from the preset file shared memory, and uploaded to the file server for storage; a transmission synchronization module is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized by using the content-recognized incremental file synchronization technology or the adaptive synchronization strategy technology according to the update operation status of the target file in the current host computer stored in the file server; an end module is used to stop the file synchronization service process when an external process stop command is received, and destroy the data in the three shared memories to end the file synchronization method.

[0112] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the file synchronization method and device.

[0113] For example, the computer program can be divided into a log module, a process startup module, a process status monitoring module, an abnormal process identification module, a device information monitoring module, a link establishment module, a link maintenance module, a shared memory module, an update status detection module, a shared memory writing module, a file loading module, a shared memory reading module, a transmission synchronization module and an end module; the specific functions of the above modules are as follows:

[0114] A log module is used to call the log recording function interface of the industrial control system, and use the log recording function interface to record the log information of subsequent file synchronization upload and download operations; a process startup module is used to use the file synchronization management process to start the file synchronization service process; wherein the file synchronization service process includes a synchronization heartbeat daemon process, a file monitoring process and a file update process; a process status monitoring module is used to monitor the process running status; an abnormal process identification module is used to identify abnormal processes; a device information monitoring module is used to use the file synchronization management process to create a device physical information monitoring thread, and use the device physical information monitoring thread to detect the physical device information of the current host computer in real time; a link building module is used to build a link between the host computer and the file server; a link maintenance module is used to use the file synchronization heartbeat daemon process to maintain the on-off status of the main network link and the backup network link; a shared memory module is used to use the file synchronization heartbeat daemon process to open up and initialize three shared memories in the memory; wherein the three shared memories include information shared memory, message shared memory and file shared memory; an update status detection module is used to detect the update status of the target file in the current host computer; A shared memory writing module is used to write the path and update operation status of the target file in the current host computer into the preset file shared memory if the update status of the target file in the current host computer changes; a file loading module is used to load the current file under the path of the target file in the previous host computer into the preset file shared memory according to the path of the target file in the current host computer written in the preset file shared memory; a shared memory reading module is used to create an upload thread; using the upload thread, the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer are read from the preset file shared memory, and uploaded to the file server for storage; a transmission synchronization module is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized by using the content-recognized incremental file synchronization technology or the adaptive synchronization strategy technology according to the update operation status of the target file in the current host computer stored in the file server; an end module is used to stop the file synchronization service process when an external process stop command is received, and destroy the data in the three shared memories to end the file synchronization method.

[0115] The file synchronization device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The file synchronization device may include, but is not limited to, a processor and a memory. Those skilled in the art may understand that the file synchronization device is only an example of a terminal device and does not constitute a limitation on the terminal device. The file synchronization device may include more or fewer components, or a combination of certain components, or different components. For example, the file synchronization device may also include an input and output device, a network access device, and a bus.

[0116] The file synchronization device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The file synchronization device may include, but is not limited to, a processor and a memory. Figure 3 The schematic diagram is merely an example of a file synchronization device and does not constitute a limitation on the file synchronization device. The file synchronization device may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the file synchronization device may also include input and output devices, network access devices, buses, etc.

[0117] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the file synchronization device, and uses various interfaces and lines to connect various parts of the entire file synchronization device.

[0118] The memory may be used to store the computer program and / or module, and the processor implements various functions of the file synchronization device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory.

[0119] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc.

[0120] In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0121] Example 4

[0122] This embodiment 4 provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the file synchronization method are implemented.

[0123] If the module / unit integrated in the file synchronization system is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0124] Based on this understanding, this embodiment 4 implements all or part of the processes in the file synchronization method described in the above embodiment 1, and can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above file synchronization method can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or preset intermediate form, etc.

[0125] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0126] It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0127] Example 5

[0128] This embodiment 5 provides a computer product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; the processor of the file synchronization device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the file synchronization device can execute the file synchronization method described in embodiment 1, which will not be repeated here.

[0129] It should be noted that a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0130] The file synchronization method, system, device, medium and product described in the present invention detect the update status of the target file in the current host computer. Once a change occurs, the file synchronization process will be immediately triggered, thereby effectively improving the real-time performance of the file synchronization method, thereby ensuring the consistency and accuracy of the data and avoiding data delay or loss; by using the preset file shared memory as the intermediate storage medium, the network bandwidth and storage space occupied during the file transmission process are effectively reduced. The file is first loaded into the shared memory, and then uploaded from the shared memory to the file server. This design reduces the frequent reading and writing of files between the disk and the network, and improves the resource utilization efficiency. According to the operation status of the file update, the synchronization strategy can be adaptively selected. For example, for the modified file, the content-recognized incremental file synchronization technology can be used to synchronize only the changed part, rather than the entire file, thereby further improving the synchronization efficiency. The method is based on the preset file shared memory and file server, so it can be easily extended to a multi-machine, multi-server environment. As long as the file server can operate normally and each host computer can access the shared memory and file server, file synchronization can be achieved. After the file is uploaded to the file server, it is stored. Even if a host computer fails, the data can be restored from the file server, ensuring the reliability of the data. The entire synchronization process is automated, and the user does not need to operate manually, reducing the possibility of human error. At the same time, since the system adopts standardized synchronization strategies and technologies, the maintenance of the system becomes simpler and more convenient.

[0131] In summary, the file synchronization method and related devices described in the present invention realize an efficient, reliable and flexible file synchronization method by real-time detection of file update status, utilization of file shared memory and file server, adaptive selection of synchronization strategy, etc., which has important practical significance for application scenarios that require frequent file synchronization.

[0132] The above embodiment is only one of the implementation methods that can realize the technical solution of the present invention. The scope of protection claimed by the present invention is not limited only to this embodiment, but also includes changes, replacements and other implementation methods that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention.

Claims

1. A file synchronization method, characterized in that: include: Check the update status of the target file in the current host computer; If the update status of the target file in the current host computer changes, the path and update operation status of the target file in the current host computer are written into the preset file shared memory; According to the path of the target file in the current host computer written in the preset file sharing memory, the current file under the path of the target file in the current host computer is loaded into the preset file sharing memory; Read the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer from the preset file sharing memory, and upload them to the file server for storage; According to the update operation status of the target file in the current host computer stored in the file server, the current file under the path of the target file in the current host computer stored in the file server is sent to the target host computer to be synchronized using content-aware incremental file synchronization technology or adaptive synchronization strategy technology.

2. A file synchronization method according to claim 1, characterized in that: The process of detecting the update status of the target file in the current host computer is as follows: Using the file monitoring process, read the monitoring path pre-specified in the monitorConfig.xml file under the / users / ems / fileServer path in the current host computer, and obtain the path of the target file in the current host computer; According to the number of paths of the target files in the current host computer, update status detection threads are created respectively; wherein, in each update status detection thread, the FileSystemWatcher control is used to detect the file update status under the path of the target file in the current host computer.

3. A file synchronization method according to claim 1, characterized in that: If the update operation status of the target file in the current host computer is a change operation, the current file under the path of the target file in the current host computer stored in the file server is sent to the target host computer to be synchronized using the content-aware incremental file synchronization technology; The process of using content-aware incremental file synchronization technology to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized is as follows: Using a preset content partitioning algorithm, the current file under the path of the target file in the current host computer and the file to be synchronized in the target host computer to be synchronized are divided into blocks respectively, and the source file block result and the file to be synchronized block result are obtained; Calculate the content hash values ​​of the source file segmentation result and the to-be-synchronized file segmentation result respectively, and obtain the source file content hash table and the to-be-synchronized file content hash table; Compare the source file content hash table with the to-be-synchronized file content hash table, filter out content blocks that differ between the source file segmentation result and the to-be-synchronized file segmentation result, and obtain the difference content blocks; The difference content blocks in the block result of the source file are replaced with the difference content blocks in the block result of the file to be synchronized, so as to obtain the update result of the file to be synchronized; Calculate the overall content hash value of the update result of the current file and the file to be synchronized under the path of the target file in the current host computer respectively, and confirm the consistency.

4. A file synchronization method according to claim 1, characterized in that: If the update operation status of the target file in the current host computer is a deletion or addition operation, the adaptive synchronization strategy technology is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized; The process of using the adaptive synchronization strategy technology to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized is as follows: Obtain the current network bandwidth of the primary network link and the backup network link between the file server and the target host computer to be synchronized; If the current network bandwidth of at least one network link among the primary network link and the backup network link is lower than the preset bandwidth threshold, the network link with the lower current network bandwidth is selected to establish the link, and the current file under the path of the target file in the current host computer is transmitted to the target host computer to be synchronized, and the current file under the path of the target file in the current host computer is used to replace the to-be-synchronized file of the target host computer to be synchronized as a whole; If the current network bandwidths of the primary network link and the backup network link are both higher than the preset bandwidth threshold, links are established simultaneously through the primary network link and the backup network link, and the current file under the path of the target file in the current host computer is transmitted to the target host computer to be synchronized using the preset flow control algorithm, and the current file under the path of the target file in the current host computer is used to replace the to-be-synchronized file of the target host computer to be synchronized as a whole.

5. A file synchronization method according to claim 1, characterized in that: In the process of sending the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized, a distributed file update strategy is used for file transmission.

6. The consistent file synchronization method according to claim 5, characterized in that: The process of file transmission using a distributed file update strategy is as follows: Get the CPU usage and memory usage of the primary file server of the file server; If the CPU occupancy rate or memory occupancy rate of the primary file server exceeds a preset occupancy rate threshold, the primary file server and the backup file server of the file server are used to perform file transfer simultaneously; wherein the primary file server is allocated less file transfer tasks than the backup file server.

7. A file synchronization system, characterized in that: include: The update status detection module is used to detect the update status of the target file in the current host computer; The shared memory writing module is used to write the path and update operation status of the target file in the current host computer into the preset file shared memory if the update status of the target file in the current host computer changes; A file loading module is used to load the current file under the path of the target file in the current host computer into the preset file sharing memory according to the path of the target file in the current host computer written in the preset file sharing memory; A shared memory reading module is used to read the update operation status of the target file in the current host computer and the current file under the path of the target file in the current host computer from the preset file shared memory, and upload them to the file server for storage; The transmission synchronization module is used to send the current file under the path of the target file in the current host computer stored in the file server to the target host computer to be synchronized according to the update operation status of the target file in the current host computer stored in the file server, using content-aware incremental file synchronization technology or adaptive synchronization strategy technology.

8. A file synchronization device, characterized in that: include: a processor suitable for executing a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the file synchronization method according to any one of claims 1 to 6 is executed.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the file synchronization method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the file synchronization method according to any one of claims 1 to 6 is implemented.

Images